[ИАД, осень 2025] Foundation models for spatial-time series. Занятие 1

Value of the Information & Strength of the Argument

The lecture provides valuable insights into the intersection of LLMs and physics-informed modeling. The argumentation is solid, building on established concepts like Hinton’s distillation and Vapnik’s privileged information to explain PINNs. The instructor’s proposal to use LLMs to generate or select models is forward-thinking and practical. However, the argumentation is somewhat informal, with digressions and a lack of formal citations, which may reduce its rigor.

Scientific Rigor, Source Quality, Title Accuracy

The lecture references several key works and concepts, including Hinton’s distillation, Vapnik’s learning theory, and the ProfIT library. However, specific sources are not formally cited, and the instructor relies on personal knowledge and experience. The title accurately reflects the content, focusing on foundation models for spatio-temporal series. The lecture is well-structured but could benefit from more explicit references to literature.

The title accurately reflects the content: the lecture introduces foundation models for spatio-temporal series, covering PINNs and state-space models.

7/10

The lecture is given by an expert in the field, referencing established concepts and specific works (e.g., Hinton's distillation, Vapnik's privileged information, PINNs). However, it lacks formal citations and is based on the instructor's personal perspective, with some informal digressions.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 02:00Introduction: course prerequisites and recommendation to read Sebastian Raschka's 'LLMs from Scratch'.
• 05:00Motivation: goal to build a system that compares many similar models, possibly using a foundation model.
• 10:00Explanation of Physics-Informed Neural Networks (PINNs) and their connection to knowledge distillation and privileged information.
• 15:00Discussion on the loss function in PINNs, balancing data fitting and teacher model fitting.
• 20:00Introduction to state-space models and their relevance for time series forecasting.
• 25:00General architecture for time series: Hankelization, dimensionality reduction, and alignment in latent space.
• 30:00Course project proposal: build a system using LLMs to select the best model for a given experiment.
• 35:00Student presentation on LLaMA architecture and its use in tool embedding.

• LLMs from Scratch by Sebastian Raschka — Recommended as a practical introduction to LLMs.
• ProfIT library — Mentioned as a repository of simple forecasting models.
• Talking GPT — Referenced as a solution based on LLaMA for tool embedding.

• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations — Foundational paper on PINNs.