Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 6 - LLM Reasoning

Item: Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 6 - LLM Reasoning
Rating: 4
Author: PostSingularity Institute

by Afshine Amidi and Shervine Amidi

Formal & Physical Sciences AI — Models & Research UYQArtificial intelligence (AI)UYQMMachine learning

🎙 Afshine Amidi and Shervine Amidi 👥 1.2M 📅 November 14, 2025 ⏱ 107 min 👁 71K 📄 lecture 🧭 2026-08-05

Available in: English (current) Français

Keywords

reasoningchain-of-thoughtRLHFGRPOPPOscalingDeepSeek R1DAPOpass@kbenchmarks

Summary

This lecture from Stanford’s CME295 course focuses on LLM reasoning. It begins by contrasting vanilla LLMs with reasoning models, highlighting limitations such as limited reasoning, static knowledge, and lack of action. The core idea is to train models to generate chain-of-thought reasoning before answering, enabling them to solve complex problems. The lecture covers key concepts: reasoning models, benchmarks (e.g., AIME, MATH), the pass@k metric, and scaling reinforcement learning (RL) for reasoning. It explains the RL setup for LLMs, comparing PPO and GRPO, and discusses practical techniques like length bias, DAPO, and Dr. GRPO. The lecture concludes with the DeepSeek R1 training recipe, emphasizing the importance of RL in improving reasoning capabilities. Throughout, the instructors provide intuition and practical insights, making it a comprehensive overview of current methods for enhancing LLM reasoning.

131 words

Critical Evaluation

The lecture provides a solid overview of LLM reasoning, effectively bridging the gap between theoretical concepts and practical training methods. The instructors, Afshine and Shervine Amidi, are experienced and present the material in a clear, structured manner. They start by recapping previous lectures on pre-training and preference tuning, which helps contextualize the new material. The definition of reasoning is appropriately nuanced, acknowledging the lack of a universally agreed-upon definition while providing a working one. The lecture covers key techniques such as chain-of-thought prompting, RL fine-tuning with PPO and GRPO, and recent advancements like DAPO and Dr. GRPO. The comparison between PPO and GRPO is particularly valuable, as it clarifies the trade-offs between these methods. The discussion of benchmarks and metrics like pass@k is practical and relevant. The DeepSeek R1 recipe provides a concrete example of how these techniques are applied in practice. However, the lecture is not without limitations. Some concepts are simplified for pedagogical purposes, and the instructors occasionally gloss over technical details. For instance, the explanation of the GRPO loss function is brief, and the mathematical derivations are not fully elaborated. Additionally, the lecture focuses heavily on math and coding reasoning, with less emphasis on other domains. The sources cited are primarily from the course syllabus and Stanford’s website, which are authoritative but not exhaustive. Overall, the lecture is a valuable resource for anyone interested in understanding and implementing LLM reasoning, but it should be supplemented with primary research papers for a deeper understanding.

246 words

Title / Content Match

The title accurately reflects the content: a lecture on LLM reasoning, covering reasoning models, RL training, and scaling.

Quality & Reliability

8/10

Lecture from Stanford University, presented by adjunct lecturers, covering established and recent research (e.g., GRPO, DAPO, DeepSeek R1). The content is well-structured, references key papers, and includes practical training recipes. However, it is a lecture, not peer-reviewed, and some claims may be simplified for pedagogical purposes.

Chapters

Cited Sources

CME295 Syllabus — Course syllabus and schedule.
Stanford Graduate Education — Information about Stanford's graduate programs.
Course Playlist — Playlist of all lectures in the course.

Concurring Sources

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — The lecture's discussion of chain-of-thought aligns with this paper.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — The lecture's coverage of DeepSeek R1 recipe is based on this paper.
GRPO: Group Relative Policy Optimization — The lecture's explanation of GRPO is consistent with this paper.

Dissenting Sources

No discordant sources identified — The lecture does not contradict established research; it is a synthesis of recent work.

Contribution & Novelties

The lecture provides a comprehensive and up-to-date overview of LLM reasoning, synthesizing recent research (2024-2025) into a coherent framework. It explains the shift from vanilla LLMs to reasoning models, emphasizing the role of RL in training. The comparison between PPO and GRPO is particularly insightful, highlighting the advantages of GRPO for reasoning tasks. The lecture also covers practical techniques like length bias and DAPO, and concludes with the DeepSeek R1 recipe, offering a concrete training pipeline. This makes it a valuable resource for practitioners.

Pour aller plus loin :

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — The foundational paper on chain-of-thought prompting.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — The paper describing the DeepSeek R1 training recipe.
GRPO: Group Relative Policy Optimization — The paper introducing GRPO.
DAPO: An Open-Source LLM Reinforcement Learning System at Scale — The paper on DAPO.
PPO: Proximal Policy Optimization Algorithms — The original PPO paper.

156 words

Radar Profile

The radar profile shows high scores in information quantity, quality, and reliability, with a slightly lower technical depth. This indicates a well-rounded lecture that is both informative and trustworthy, though it may not delve into the most advanced mathematical details.

Reliability 8/10