16-899 Adaptive Control and Reinforcement Learning (Spring 2020)

(Last Update: 5/21/2020)
Time: Tuesday and Thursday 9:00am - 10:20am
Location: https://cmu.zoom.us/j/753520524

Instructor: Changliu Liu, cliu6@andrew.cmu.edu
Instructor office hours: Tuesday 10:30am - 11:30am, https://cmu.zoom.us/j/184522962

Teaching Assistant: Saumya Saxena (saumyas@andrew.cmu.edu) and Yeeho Song (yeehos@andrew.cmu.edu)
TA office hours: Wednesday 3pm - 4pm and Friday 12pm - 1pm, https://cmu.zoom.us/j/184522962

Piazza: https://piazza.com/class/k54pll5057h79m
Gradescope: https://www.gradescope.com/courses/79834

Course Description

This course will discuss algorithms that learn and adapt to the environment. This course is directed to students—primarily graduate although talented undergraduates are welcome as well—interested in developing adaptive software that makes decisions that affect the world. This course will discuss adaptive behaviors both from the control perspective and the learning perspective.

Key Topics

optimal control, model predictive control, iterative learning control, adaptive control, reinforcement learning, imitation learning, approximate dynamic programming, parameter estimation, stability analysis.

Course Goal

To familiarize the students with algorithms that learn and adapt to the environment. To provide a theoretical foundation for adaptable algorithm.

Syllabus

syllabus

Prerequisite

As an advanced course, familiarity with basic ideas from control theory, robotics, probability, machine learning will be helpful. Useful courses to have taken in advance include Statistical Techniques in Robotics, Artificial Intelligence, and Kinematics, Dynamics, and Control. As the course will be project driven, prototyping skills including C, C++, Python, and Matlab will also be important. Creative thought and enthusiasm are required.

Assessment Structure

Homework - 20%
Midterm - 20%
Final project - 10% (proposal) + 30% (report) + 10% (presentation)
Participation - 10%

Student Final Projects (Selected)

Alvin Shek and Tom Scherlis.
Highly Dynamic Quadcopter Control For Drone Racing.
[Slides] [Report]

Angelos Mavrogiannis.
Vehicle Trajectory Generation via Human Driver Behavior Classification.
[Slides]

Ruixuan Liu.
Human Motion Prediction with Adaptable RNN.
[Slides] [Report]

Yingjia Hu.
Imitate Lunar Lander Based on DAGGER.
[Slides]

References

Optional textbook:
Goodwin. Adaptive Filtering, Prediction, and Control.
Bertsekas. Optimal Control and Reinfocement Learning.
Anderson and Moore. Optimal Control, Linear Quadratic Methods.
Borrelli. Predictive control for linear and hybrid systems.

Related course:
https://www.cc.gatech.edu/~bboots3/ACRL-Spring2019/
http://www.cs.cmu.edu/afs/cs/project/ACRL/www/SyllabusAdaptiveControlAndReinforcement.htm