10703 (Spring 2018): Deep RL and Control - Lecture Schedule

Tentative Lecture Schedule

Jan 17, Introduction [pdf]
Reading: Sutton & Barto, Chapters 1,2
Jan 22, Markov decision processes (MDPs) [pdf]
Reading: Sutton & Barto, Chapter 3
Jan 24, Solving known MDPs: Dynamic Programming [pdf]
Reading: Sutton & Barto, Chapter 4
Jan 29, Monte Carlo learning [pdf]
Reading: Sutton & Barto, Chapter 5
Jan 31, Temporal difference learning [pdf]
Reading: Sutton & Barto, Chapter 6
Feb 5, VF approximation, MC, TD with VF approximation. [pdf]
Reading: Sutton & Barto, Chapter 9
Feb 7, VF approximation, Deep Learning, Neural Nets. [pdf]
Reading: Deep Learning book, GBC, Chapter 6
Feb 12, VF approximation, Deep Learning, Convnets, Optimization. [pdf]
Reading: Deep Learning book, GBC, Chapter 9
Feb 14, Deep Q-Learning, Double Q-Learning, Replay Memory [pdf]
Reading: Relevant papers (see lecture slides)
Feb 19, Monte carlo tree search [pdf]
Reading: Sutton & Barto, Chapter 8
Feb 20, Feb 26, Policy Gradient I, Policy Gradient II [pdf], [pdf]
Reading: Sutton & Barto, Chapter 13
Feb 28, Continuous Actions, Variational Autoencoders, multimodal stochastic policies [pdf]
Reading: Deep Learning book, GBC, Chapter 20.10
March 5, No class
March 7, Imitation Learning I: Behavior Cloning, DAGGER, Structured Prediction [pdf]
Reading:
- Bagnell, An Invitation to Imitation
- Nguyen, Imitation Learning with Recurrent Neural Networks
- Bojarski et al., End to End Learning for Self-Driving Cars
- Guo et al., Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning
- Rouhollah et al., Learning real manipulation tasks from virtual demonstrations using LSTM
- Ross et al., Learning Monocular Reactive UAV Control in Cluttered Natural Environments
- Ross et al., A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
March 12-14, Spring Break
March 19, Imitation Learning II: Inverse RL, MaxEnt IRL [pdf]
Reading:
- Ziebart et al., Navigate Like a Cabbie: Probabilistic Reasoning from Observed Context-Aware Behavior
- Abbeel et al., Apprenticeship Learning via Inverse Reinforcement Learning
- Ho et al., Model-Free Imitation Learning with Policy Optimization
- Finn et al., Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
- Ziebart et al., Maximum Entropy Inverse Reinforcement Learning
- Ziebart et al., Human Behavior Modeling with Maximum Entropy Inverse Optimal Control
- Finn et al., Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
March 21, Optimal control, trajectory optimization [pdf]
Reading:
- Tassa et al., Synthesis and Stabilization of Complex Behaviors through Online Trajectory Optimization
March 26, Guest Lecture
March 28, Optimal control, trajectory optimization, part II [pdf]
Reading:
- Tassa et al., Synthesis and Stabilization of Complex Behaviors through Online Trajectory Optimization
April 2, 4, Learning Local models, TRPO, Imitating Optimal Controllers [pdf]
Reading:
- Watter et al., Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
- Levine et al., Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics
- Levine et al., Guided Policy Search
- Levine et al., End-to-End Training of Deep Visuomotor Policies
- Kumar et al., Learning Dexterous Manipulation Policies from Experience and Imitation
April 9,11 End-to-end Model Based Reinforcement Learning [pdf]
Reading:
- Mishra et al., Prediction and Control with Temporal Segment Models
- Lillicrap et al., Continuous control with deep reinforcement learning
- Heess et al., Learning Continuous Control Policies by Stochastic Value Gradients
April 16, Guest Lecture
April 16, Exploration and Exploitation [pdf]
Reading: Sutton & Barto, Chapter 2
April 23, Student Project Presentations
April 25, Hierarchical RL and Tranfer Learning [pdf]
April 30, Memory Augmented RL [pdf]

[ Home | Assignments | Lecture Schedule | ]

10703 (Spring 2018): Deep RL and Control || http://www.cs.cmu.edu/~rsalakhu/10703/