Lecture 4: MIT 6.832 Underactuated Robotics (Spring 2022) | "Dynamic Programming II"

The lecture provides a rigorous and insightful derivation of the Hamilton-Jacobi-Bellman equation, bridging the gap between discrete and continuous time dynamic programming. Tedrake’s pedagogical approach is effective, starting with a review of discrete-time concepts and then smoothly transitioning to the continuous-time formulation. The mathematical derivations are clear and well-motivated, with careful attention to notation and assumptions. The connection to physics, particularly Hamiltonian mechanics, adds depth and helps students appreciate the fundamental nature of the HJB equation. The lecture also addresses practical challenges, such as the non-smoothness of value functions and the computational difficulties in solving HJB equations, which is valuable for students aiming to apply these methods. The use of examples, such as the pendulum and double integrator, helps illustrate the concepts. The lecture is well-structured and suitable for an advanced undergraduate or graduate-level audience. The only minor criticism is that the lecture could benefit from more concrete numerical examples or visualizations to reinforce the theoretical concepts. Overall, this is an excellent lecture that provides a solid foundation for understanding continuous-time optimal control.

The title accurately reflects the content, which focuses on dynamic programming in continuous time, building on the previous lecture.

9/10

Lecture from MIT OpenCourseWare by a renowned professor in robotics, with rigorous mathematical derivations and references to course materials. The content is well-structured and aligns with established theory in optimal control and dynamic programming.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction and recap of discrete-time dynamic programming
• 05:00Motivation for continuous-time formulation and limitations of discrete methods
• 10:00Derivation of the Hamilton-Jacobi-Bellman equation from discrete-time Bellman equation
• 15:00Intuition behind the HJB equation and its connection to physics
• 20:00Discussion on the challenges of solving HJB equations, including non-smooth solutions
• 25:00Introduction to viscosity solutions and level set methods
• 30:00Examples and applications of HJB in underactuated robotics
• 35:00Preview of future topics: LQR and trajectory optimization

• Lecture slides — Slides used in the lecture, providing visual aids and additional details.

• Underactuated Robotics textbook — Companion textbook by Russ Tedrake, covering related topics in depth.