16-899 Adaptive Control and Reinforcement Learning (Fall 2022)

(Last Update: 8/26/2022)

Time: Monday and Wednesday 10:10-11:30
Location: Wean Hall 2302

Instructor: Changliu Liu, cliu6@andrew.cmu.edu

Canvas: https://canvas.cmu.edu/courses/30981

Course Description

This course will discuss algorithms that learn and adapt to the environment. This course is directed to students—primarily graduate although talented undergraduates are welcome as well—interested in developing adaptive software that makes decisions that affect the world. This course will discuss adaptive behaviors both from the control perspective and the learning perspective.

Key Topics

optimal control, model predictive control, iterative learning control, parameter estimation, indirect adaptive control, reinforcement learning, stability analysis, safety analysis.

Course Goal

To familiarize the students with algorithms that learn and adapt to the environment. To provide a theoretical foundation for adaptable algorithm.

Prerequisite

As an advanced course, familiarity with basic ideas from control theory, robotics, probability, machine learning will be helpful. Useful courses to have taken in advance include Statistical Techniques in Robotics, Artificial Intelligence, and Kinematics, Dynamics, and Control. As the course will be project driven, prototyping skills including Matlab, Python, C, and C++ will also be important. Creative thought and enthusiasm are required.

Assessment Structure

Homework - 50%
Final project - 5% (proposal) + 25% (report) + 10% (presentation)
Participation - 10%

References

Optional textbook:
Goodwin. Adaptive Filtering, Prediction, and Control.
Bertsekas. Optimal Control and Reinfocement Learning.
Anderson and Moore. Optimal Control, Linear Quadratic Methods.
Borrelli. Predictive control for linear and hybrid systems.
Sutton and Barto. Reinforcement Learning: An Introduction.