Stanford CS329A Self-Improving AI Agents | Part 7 | Self-Improvement and Deep Research Agents

The lecture provides a rigorous and detailed examination of self-improvement through search in AI agents. It is grounded in well-known research papers (AlphaCode, AlphaCode2, Search-O1, Search-R1) and offers a clear explanation of the methodologies, including pre-training, fine-tuning, large-scale sampling, filtering, clustering, and selection. The instructor demonstrates a deep understanding of the subject, providing insights into the bottlenecks and trade-offs involved. The content is technically rich, with specific details on model sizes, sample counts, and benchmark results. The lecture also encourages critical thinking by asking students to consider sources of variance in performance across different contests. The sources cited are the official course materials and the referenced papers, which are appropriate for an academic lecture. The title accurately reflects the content, and the lecture is well-structured, with a logical flow from code generation to deep research agents. The main limitation is that the lecture assumes prior knowledge of LLMs and test-time compute, making it less accessible to a general audience. However, for its intended audience, it is an excellent resource.

The title accurately reflects the content, which covers self-improvement through search and deep research agents.

8/10

Lecture from Stanford CS329A, presented by an adjunct professor with deep expertise in LLMs. Content is based on peer-reviewed research (AlphaCode, AlphaCode2, Search-O1, Search-R1) and includes technical details. The lecture is well-structured and provides critical analysis of the methods.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction to self-improvement through search, overview of AlphaCode and deep research agents.
• 02:00AlphaCode's approach: pre-training on GitHub and CodeContests, large-scale sampling, filtering, and clustering.
• 10:00Discussion of AlphaCode's performance on Codeforces, ranking top 54% among participants.
• 20:00Analysis of how solve rate scales with sample budget and the role of selection and clustering.
• 30:00Introduction to AlphaCode2, fine-tuning Gemini Pro with a learned scoring model, achieving 85th percentile.
• 40:00Transition to deep research agents, introduction of Search-O1 and its uncertainty-based search triggering.
• 50:00Comparison of Search-O1 with standard and agentic RAG on GPQA and multi-hop QA benchmarks.
• 1:00:00Comparison of Search-O1's prompting-based approach with Search-R1's reinforcement-learning-based approach.

• CS329A Course Website — Course syllabus and schedule for CS329A Self-Improving AI Agents.
• Stanford Online CS329A Course Page — Official Stanford Online course page for CS329A.
• Agentic AI Professional Education Program — Stanford professional education program related to agentic AI.
• Course Playlist — YouTube playlist for the CS329A course lectures.

• AlphaCode paper — The original AlphaCode paper, which the lecture discusses in detail.
• AlphaCode2 paper — The AlphaCode2 paper, which the lecture discusses as an improvement over AlphaCode.

• Search-O1 paper