Lecture 3: MIT 6.832 Underactuated Robotics (Spring 2022) | "Dynamic Programming I"

This lecture provides a rigorous introduction to dynamic programming in the context of optimal control, specifically tailored for underactuated robotics. The instructor, a leading expert in the field, delivers the content with clarity and depth, making it suitable for advanced undergraduate or graduate students. The lecture’s strength lies in its careful progression from a concrete problem (pendulum swing-up) to a general framework (dynamic programming), illustrating the necessity of a more sophisticated approach. The derivation of the bang-bang policy for the double integrator is particularly well-executed, combining physical intuition with mathematical analysis. The discussion of the cost-to-go function and its non-smoothness highlights subtle issues that are often glossed over in introductory treatments. The connection to reinforcement learning is appropriately drawn, acknowledging the shared foundations while noting differences in terminology and emphasis. The lecture does not shy away from technical details, but it also provides intuitive explanations that aid understanding. The use of the phase portrait is effective in visualizing the optimal policy. The lecture is well-structured, with clear objectives and a logical flow. The instructor’s teaching style is engaging, with occasional humor, which helps maintain attention. The content is accurate and up-to-date, reflecting current research perspectives. The lecture does not include any apparent biases or unsupported claims. The sources cited are primarily the instructor’s own textbook and other standard references in the field, which are appropriate. The adéquation between the title and content is excellent, as the lecture indeed focuses on dynamic programming. Overall, this is an excellent lecture that provides a solid foundation for further study in optimal control and reinforcement learning.

The title accurately reflects the content: a lecture on dynamic programming in the context of underactuated robotics.

9/10

The lecture is part of MIT's official OpenCourseWare, presented by a recognized expert in the field. The content is rigorous, mathematically grounded, and includes references to foundational concepts. The presentation is clear and well-structured, with a focus on both theoretical foundations and practical implications.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction and recap of pendulum swing-up problem
• 02:00Discussion of limitations of simple control laws
• 05:00Introduction to cost functions and optimal control
• 10:00Comparison between optimal control and reinforcement learning
• 15:00Formulation of the double integrator problem
• 20:00Derivation of bang-bang policy for minimum-time control
• 25:00Phase portrait analysis and optimal cost-to-go
• 30:00Discussion of non-smoothness in cost-to-go
• 35:00Transition to discretized dynamic programming

• Underactuated Robotics (course textbook) — The lecture is based on the instructor's textbook, which is freely available online.

• Underactuated Robotics (course textbook) — The lecture is based on the instructor's textbook, which is freely available online.