Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 7 - Agentic LLMs

This lecture provides a comprehensive and well-structured overview of techniques for making LLMs more useful in real-world applications. The instructors, Afshine and Shervine Amidi, are adjunct lecturers at Stanford, and their academic background lends credibility to the content. The lecture is part of a formal course, which ensures a logical progression and depth of coverage.

The content on RAG is particularly strong. The instructors clearly explain the motivation behind RAG, addressing the limitations of static knowledge cutoffs and the impracticality of continuous fine-tuning. They present a balanced view of the challenges, such as context length limits, performance degradation with irrelevant information (needle-in-a-haystack), and cost considerations. The discussion of retrieval techniques is thorough, covering both embedding-based methods (SBERT, bi-encoders) and heuristic methods (BM25), as well as advanced techniques like HyDE and contextual retrieval. The inclusion of evaluation metrics (NDCG, MRR) is valuable for practitioners.

The section on tool calling and agents is equally informative. The instructors explain how LLMs can be extended to interact with external tools, and they introduce the ReAct framework, which combines reasoning and acting. The mention of the Model Context Protocol (MCP) is timely, as it is an emerging standard for tool integration. The safety considerations at the end are important and show a responsible approach.

The lecture is well-paced and includes helpful examples and analogies. The instructors also address student questions, which enhances understanding. The use of slides and diagrams aids comprehension.

One minor weakness is that the lecture does not delve deeply into the mathematical underpinnings of some techniques, but this is appropriate for a course that appears to target a broad audience. The focus is on practical application rather than theoretical derivation.

Overall, this is an excellent lecture that provides a solid foundation for anyone looking to implement RAG and agentic workflows with LLMs. The information is accurate, up-to-date, and presented in an engaging manner. The sources cited are credible, and the instructors are knowledgeable. The lecture earns a high rating for its quality and educational value.

The title accurately reflects the content, focusing on agentic LLMs and covering RAG, tool calling, and agents as promised.

9/10

Lecture from Stanford University by experienced instructors, covering established techniques (RAG, tool calling, ReAct) with references to recent research and practical considerations. High credibility due to academic affiliation and structured presentation.

• 00:00Introduction
• 06:38RAG overview
• 27:39Similarity search with SBERT and bi-encoders
• 34:25Heuristic search with BM25
• 37:54HyDE and contextual retrieval
• 41:00Prompt caching
• 45:24Re-ranking with cross-encoders
• 47:49Retrieval evaluation with NDCG, MRR
• 59:28Tool calling
• 1:26:22Tool selection
• 1:29:17Model Context Protocol (MCP)
• 1:31:56Agents with ReAct
• 1:42:16Safety and closing thoughts

• CME295 Course Syllabus — Course schedule and syllabus for following along with the lecture.
• Stanford Online Graduate Education — Information about Stanford's graduate programs.
• CME295 Course Playlist — Playlist of all lectures in the course.

• Retrieval-Augmented Generation for Large Language Models: A Survey — Provides a comprehensive overview of RAG techniques, consistent with the lecture's content.
• ReAct: Synergizing Reasoning and Acting in Language Models — The original ReAct paper, which the lecture references for agentic workflows.