Reinforcement Learning 2026 - Session 1

This is the first session of a Reinforcement Learning (RL) course, taught in Persian by an instructor from the Robust and Interpretable Machine Learning Lab. The session begins with course logistics: there will be 8 problem sets (7 points total), 2 projects (4 points), a midterm and a final exam (each 3 points), and a total of 14 late days. The instructor then introduces RL through a robotics example: a robot with 7 degrees of freedom must pick objects from a tray using a camera. He contrasts a low-level controller (joint movements) with a high-level controller (deciding where to place the gripper). He explains why a physics-based approach is hard, and why supervised learning is not scalable due to the need for labeled data. Instead, he proposes a data-centric approach where the robot collects its own data by trying actions and receiving success/failure labels. This leads to a loop: the agent interacts with the environment, collects data, and improves its policy. He discusses the challenges of RL, such as the sparse reward problem, delayed rewards, credit assignment, and non-i.i.d. data. He formalizes RL by contrasting it with supervised learning: an agent interacts with an environment, receives rewards, and aims to maximize the return (discounted sum of rewards). The lecture ends with a Q&A session.