|
Schedule (tentative)
Here is a tentative syllabus for the course. Readings will be filled in over time.
Recommended Textbooks
RL: Reinforcement Learning: An Introduction. Richard S. Sutton and Andrew G. Barto. Second Edition, MIT Press, Cambridge, MA, 2018. Available online.
CV: Computer Vision: Algorithms and Applications 2nd Edition. Richard Szeliski, Microsoft Research. Available online.
RKH: Robotic Systems. Kris Hauser. Draft available online.
MR: Modern Robotics: Mechanics, Planning, and Control. Frank C. Park, Kevin M. Lynch. Cambridge University Press. Available Online.
| Date | Topic | Slides | Readings | Assignments |
| Aug 24 | Introduction | pdf | |
|
| Part I | Review | | | |
| Aug 26 | Computer Vision Review | geom.pdf, geom.pptx | CV Chapters 7, 8, 11 |
|
| Aug 31 | Computer Vision Review | cnn.pdf, cnn.pptx | CV Chapters 5, 6 |
|
| Sep 2 | Robotics Review | pdf | RKH Chapters 5, 6, 10, 17 | Assignment 1
|
| Sep 7 | Robotics Review | | |
|
| Sep 9 | MDP Review | pdf | RL Chapter 3, 4 |
|
| Sep 14 | MDP Review [Dynamic Programming] | pdf, notes | RL Chapter 3, 4 |
|
| Sep 16 | MDP Review [Monte Carlo, TD,
Model Free Control] | notes | RL Chapter 5, 6
DQN, Rainbow DQN | Assignment 1 Due
Sept 16, 2021 11:59 PM |
| Sep 21 | Q-Learning | key, pdf | DDPG,
BCQ | |
| Sep 23 | Policy Gradients | notes,
pdf, key | PPO | Assignment 2
|
| Part II | Alternatives to Solving Unknown MDPs | | | |
| Sep 28 | Model Building | key, pdf | ME-TRPO,
PETS | Projects |
| Sep 30 | Model Building | | Deep Visual Foresight |
|
| Oct 5 | Imitation Learning | key, pdf | DAgger,
Implicit BC |
| Oct 7 | Inverse RL | key, pdf | Inverse RL,
GAIL | Assignment 2 Due
Oct 7, 2021 11:59 PM |
| Oct 12 | Self-supervision | key, pdf | Self-supervised Grasping | Project Proposal due
Oct 12, 2021 11:59 PM |
| Oct 14 | Exploration | | Planning to Explore |
|
| Oct 19 | Sim to Real | key, pdf | Rapid Motor Adaptation |
|
| Oct 21 | Social Learning | pdf | Navigation Subroutines |
|
| Oct 26 | Social Learning | pdf | Value Learning from Videos,
Rewards from Videos | Assignment 3 |
| Oct 28 | Hierarchies | key, pdf | FuN |
|
| Nov 2 | Inductive Bias | key, pdf | OptNet |
|
| Part III | Case Studies | | | |
| Nov 4 | Navigation | key, pdf
key, pdf | Neural Topological SLAM | Progress Report due
Nov 4, 2021 11:59 PM |
| Nov 9 | Mobile Manipulation | key, pdf | ReLMoGen | Assignment 3 Due
Nov 9, 2021 11:59 PM |
| Nov 11 | Manipulation | key, pdf | DexNet 2.0 |
|
| Nov 16 | Manipulation | key, pdf | Re-grasping using Touch |
|
| Part IV | Perspectives | | | |
| Nov 18 | Lessons from Cognitive Science
and Psychology | pptx, pdf | 6 Lessons | |
| Nov 23 | Fall Break (No class) |
| | |
| Nov 25 | Fall Break (No class) |
| | |
| Nov 30 | Data vs Algorithms | key, pdf | The Bitter Lesson+A Better Lesson |
|
| | | | Projects Due
Dec 1, 2021 11:59 PM |
| Dec 2 | Project Presentations | key, pdf |
| |
| Dec 7 | Project Presentations |
| |
|
PILCO: A model-based and data-efficient approach to policy search
Marc Deisenroth and Carl Rasmussen
ICML 2011
Deep visual foresight for planning robot motion
Chelsea Finn and Sergey Levine
ICRA 2017
Deep reinforcement learning in a handful of trials using probabilistic dynamics models
Kurtland Chua, Roberto Calandra, Rowan McAllister, Sergey Levine
NeurIPS 2018
Benchmarking model-based reinforcement learning
Eric Langlois, Shunshi Zhang, Guodong Zhang, Pieter Abbeel, Jimmy Ba
arXiv preprint arXiv:1907.02057 2019
A reduction of imitation learning and structured prediction to no-regret online learning
Stephane Ross, Geoffrey Gordon, Drew Bagnell
AISTATS 2011
End-to-end training of deep visuomotor policies
Sergey Levine, Chelsea Finn, Trevor Darrell, Pieter Abbeel
JMLR 2016
Maximum entropy inverse reinforcement learning.
Brian Ziebart, Andrew Maas, J Bagnell, Anind Dey
AAAI 2008
Apprenticeship learning via inverse reinforcement learning
Pieter Abbeel and Andrew Ng
ICML 2004
The development of embodied cognition: Six lessons from babies
Linda Smith and Michael Gasser
Artificial life 2005
Building machines that learn and think like people
Brenden Lake, Tomer Ullman, Joshua Tenenbaum, Samuel Gershman
Behavioral and brain sciences 2017
Playing atari with deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller
arXiv preprint arXiv:1312.5602 2013
Scalable trust-region method for deep reinforcement learning using kronecker-factored approximation
Yuhuai Wu, Elman Mansimov, Roger Grosse, Shun Liao, Jimmy Ba
NeurIPS 2017
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra
ICLR 2016
Off-policy deep reinforcement learning without exploration
Scott Fujimoto, David Meger, Doina Precup
ICML 2019
Learning agile and dynamic motor skills for legged robots
Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, Marco Hutter
Science Robotics 2019
Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours
Lerrel Pinto and Abhinav Gupta
ICRA 2016
Learning synergies between pushing and grasping with self-supervised deep reinforcement learning
Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, Thomas Funkhouser
IROS 2018
Planning to Explore via Self-Supervised World Models
Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, Deepak Pathak
ICML 2020
Curiosity-driven exploration by self-supervised prediction
Deepak Pathak, Pulkit Agrawal, Alexei Efros, Trevor Darrell
ICML 2017
Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations
Shuran Song, Andy Zeng, Johnny Lee, Thomas Funkhouser
RAL 2020
Learning Navigation Subroutines from Egocentric Videos
Ashish Kumar, Saurabh Gupta, Jitendra Malik
CoRL 2019
Differentiable MPC for end-to-end planning and control
Brandon Amos, Ivan Jimenez, Jacob Sacks, Byron Boots, J Kolter
NeurIPS 2018
Differntiable Spatial Planning using Transformers
Anonymous Anonymous
Open Review 2020
Neural Topological SLAM for Visual Navigation
Devendra Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, Saurabh Gupta
CVPR 2020
More than a feeling: Learning to grasp and regrasp using vision and touch
Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wenzhen Yuan, Jitendra Malik, Edward Adelson, Sergey Levine
RAL 2018
Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics
Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Ojea, Ken Goldberg
RSS 2017
Semantic Visual Navigation by Watching YouTube Videos
Matthew Chang, Arjun Gupta, Saurabh Gupta
NeurIPS 2020
Algorithms for inverse reinforcement learning.
Andrew Ng and Stuart Russell
ICML 2000
Feudal reinforcement learning
Peter Dayan and Geoffrey Hinton
NeurIPS 1993
Unsupervised perceptual rewards for imitation learning
Pierre Sermanet, Kelvin Xu, Sergey Levine
Robotics: Science and Systems 2017
Agile autonomous driving using end-to-end deep imitation learning
Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos Theodorou, Byron Boots
Robotics: Science and Systems 2018
Cognitive mapping and planning for visual navigation
Saurabh Gupta, Varun Tolani, James Davidson, Sergey Levine, Rahul Sukthankar, Jitendra Malik
International Journal of Computer Vision 2019
Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation
Mohit Sharma, Jacky Liang, Jialiang Zhao, Alex LaGrassa, Oliver Kroemer
CoRL 2020
The Bitter Lesson
Rich Sutton
Blogpost 2019
A Better Lesson
Rodney Brooks
Blogpost 2019
RMA: Rapid motor adaptation for legged robots
Ashish Kumar, Zipeng Fu, Deepak Pathak, Jitendra Malik
Robotics: Science and Systems 2021
Proximal policy optimization algorithms
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov
arXiv preprint arXiv:1707.06347 2017
Rainbow: Combining improvements in deep reinforcement learning
Matteo Hessel, Joseph Modayil, Hado Van, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, David Silver
AAAI 2018
Model-ensemble trust-region policy optimization
Thanard Kurutach, Ignasi Clavera, Yan Duan, Aviv Tamar, Pieter Abbeel
ICLR 2018
Generative adversarial imitation learning
Jonathan Ho and Stefano Ermon
NeurIPS 2016
Implicit Behavioral Cloning
Pete Florence, Corey Lynch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, Jonathan Tompson
CoRL 2021
Differentiable Spatial Planning using Transformers
Devendra Chaplot, Deepak Pathak, Jitendra Malik
ICML 2021
Feudal networks for hierarchical reinforcement learning
Alexander Vezhnevets, Simon Osindero, Tom Schaul, Nicolas Heess, Max Jaderberg, David Silver, Koray Kavukcuoglu
ICML 2017
Optnet: Differentiable optimization as a layer in neural networks
Brandon Amos and J Kolter
ICML 2017
Learning Generalizable Robotic Reward Functions from“ In-The-Wild” Human Videos
Annie Chen, Suraj Nair, Chelsea Finn
RSS 2021
Relmogen: Leveraging motion generation in reinforcement learning for mobile manipulation
Fei Xia, Chengshu Li, Roberto Mart{'i}n-Mart{'i}n, Or Litany, Alexander Toshev, Silvio Savarese
ICRA 2021
|