Stanford CME296 Diffusion & Large Vision Models | Spring 2026 | Lecture 2 - Score matching

This lecture provides a comprehensive and rigorous introduction to score matching, a fundamental concept in modern generative modeling. The instructors, Afshine and Shervine Amidi, are adjunct lecturers at Stanford, and their expertise is evident in the clear and logical progression of the material. The lecture builds on the previous session’s coverage of DDPM, establishing a strong foundation by contrasting the two paradigms. The mathematical derivations are thorough, with careful attention to the tractability of the score function and the rationale behind denoising score matching. The use of Gaussian distributions as a tractable proxy is well-explained, and the limitations of the approach are honestly addressed, leading to the introduction of noise conditional score networks and annealed Langevin dynamics. The connection between score matching and DDPM is elegantly drawn, unifying the two perspectives. The extension to continuous-time SDEs is a natural and important step, as it provides a general framework for understanding diffusion models. The lecture also covers practical aspects such as training and inference, including the Euler-Maruyama method and the probability flow ODE, and introduces DPM-Solver as an efficient inference technique. The content is highly accurate and aligns with the established literature, including key papers by Song et al. and Ho et al. The lecture is well-paced, with interactive questions that enhance understanding. The only minor weakness is that some derivations are skipped due to time constraints, but the instructors acknowledge this and provide references. Overall, this is an excellent lecture that would benefit both students and practitioners seeking a deep understanding of score-based generative models.

The title accurately reflects the content, which focuses on score matching as a generative modeling paradigm.

9/10

Lecture from Stanford University by experienced instructors, covering established mathematical foundations of score matching and diffusion models. The content is rigorous, well-structured, and aligns with published research in the field.

• 00:00Introduction
• 04:33Motivation behind score matching
• 14:30Lanvegin sampling
• 18:43Score estimation
• 19:50Implicit score matching, sliced score matching
• 20:56Score of a Gaussian distribution
• 29:58Denoising score matching
• 40:44Limitations of DSM
• 49:49Noise conditional score networks
• 52:49Annealed Langevin dynamics
• 58:09Parallel between DDPM and score
• 1:04:37Continuous derivation
• 1:14:31SDE formulation
• 1:22:54Training
• 1:24:29Reverse SDE
• 1:28:46Inference with Euler-Maruyama
• 1:30:32PF-ODE
• 1:41:24DPM-Solver

• Course syllabus — Official course syllabus for CME296, providing schedule and materials.
• Course page — Stanford Online course page with details and enrollment information.
• Stanford Graduate Education — Information about Stanford's graduate programs.
• Course playlist — YouTube playlist containing all lectures of CME296.

• Score-Based Generative Modeling through Stochastic Differential Equations — This paper provides the theoretical foundation for the SDE formulation discussed in the lecture.
• Denoising Score Matching for Score-based Generative Modeling — This paper introduces denoising score matching, a core concept covered in the lecture.
• Generative Modeling by Estimating Gradients of the Data Distribution — This paper presents noise conditional score networks and annealed Langevin dynamics, which are discussed in the lecture.