ECE598SG: Special Topics in Learning-based Robotics (Fall 2022)

Instructor: Saurabh Gupta (saurabhg)
TA: Aditya Prakash (adityap9)
Lecture Times: Tuesday / Thursday 9:30AM - 10:50AM
Lecture Location: ECEB 2015

Office Hours | Campuswire (announcements and discussion) | Gradescope (programming assignments, entry code: 4VG8ZD)

To the extent possible, you should use Campuswire for asking questions. If you have a logistics question specific to yourself, you can email the course staff at

Access to Campuswire: If you are registered for the class, you should have received a invite to Campuswire. If you haven't, please drop us an email at

Course Description

This course will introduce students to recent developments in the area of robot learning. The course will start with an overview of background material from relevant subfields: computer vision, machine learning, robotics and control theory. This will be followed by advanced techniques (model-free reinforcement learning with function approximators, model learning, model-based reinforcement learning with learned models, imitation learning, inverse reinforcement learning, self-supervised learning, exploration, hierarchies) in this area. These advanced techniques will be covered via recent research papers that develop and validate them. The course will conclude with case-studies on robotic navigation, and manipulation from recent papers. Project work as part of the course will provide a flavor of research in this new emerging area.

Learning Outcomes

After taking this course you will be able to:

  1. Be able to describe, at a high-level, the various components of a typical robotics pipeline

  2. Be able to describe Markov Decision Processes (MDPs)

  3. Be able to formulate robotics problems as MDPs

  4. Be able to describe and implement different model-free and model-based methods for solving MDPs under varying assumptions and settings (known and unknown models, low-dimensional states versus high-dimensional observations)

  5. Be able to articulate the challenges and limitations of solving robotics problems through model-free reinforcement learning

  6. Be able to describe and implement alternative learning techniques where model-free reinforcement learning is infeasible, and models aren't available

  7. Be able to describe how aforementioned learning techniques can be used to tackle problems in robotic navigation and manipulation

  8. Be able to critique recent literature in robot learning

  9. Be able to design machine learning sub-systems for robotics problems taking design consideration into account

  10. Be able to design experiments, conduct analysis, and compile experimental findings to validate different learning approaches


This is an advanced gradate course aimed at graduate students conducting research in relevant research areas. The course will be structured around research papers published within the last few years in computer vision, robotics, and machine learning. Students should be familiar with reading and critiquing research papers, and should have a basic understanding of concepts in artificial intelligence, and machine learning. Students must have taken at least one of the following (or equivalent) courses: ECE 448 / CS 440 (Introduction to Artificial Intelligence), ECE 544NA (Pattern Recognition), ECE 549 / CS 543 (Computer Vision). If you are not sure whether you meet the prerequisites, talk to the instructor after the first class or in office hours.

Recommended Textbooks

  1. Reinforcement Learning: An Introduction. Richard S. Sutton and Andrew G. Barto. Second Edition, MIT Press, Cambridge, MA, 2018. Available online.

  2. Computer Vision: Algorithms and Applications 2nd Edition. Richard Szeliski, Microsoft Research. Available online.

  3. Robotic Systems. Kris Hauser. Draft available online.

  4. Modern Robotics: Mechanics, Planning, and Control. Frank C. Park, Kevin M. Lynch. Cambridge University Press. Available Online.

Grading Scheme

Tentative and subject to change.

  1. Programming assignments: 35%. 3 MPs, to be done individually.

  2. Paper Reviews: 15%. More details here.

  3. Quizzes: 10%. 2 Quizzes through the semester. More details here.

  4. Final project: 30%. Students will engage in projects in groups of 1-3. Projects should involve investigation of relevant research questions in computer vision, robotics and machine learning. Projects could be done in simulation, or on real-robot platforms. See here for more details.

  5. Participation: 10% Participation in in-class and Campuswire discussion.


We thank the many researchers who have made their slides and course materials available.