16-899 Adaptive Control and Reinforcement Learning (Fall 2021)
(Last Update: 12/15/2021)
Time: Monday and Wednesday 10:10-11:30
Location: zoom
Instructor: Changliu Liu, cliu6@andrew.cmu.edu
Canvas: https://canvas.cmu.edu/courses/24769
Course Description
This course will discuss algorithms that learn and adapt to the environment. This course is directed to students—primarily graduate although talented undergraduates are welcome as well—interested in developing adaptive software that makes decisions that affect the world. This course will discuss adaptive behaviors both from the control perspective and the learning perspective.
Key Topics
optimal control, model predictive control, iterative learning control, parameter estimation, indirect adaptive control, reinforcement learning, stability analysis, safety analysis.
Course Goal
To familiarize the students with algorithms that learn and adapt to the environment. To provide a theoretical foundation for adaptable algorithm.
Course Material
Prerequisite
As an advanced course, familiarity with basic ideas from control theory, robotics, probability, machine learning will be helpful. Useful courses to have taken in advance include Statistical Techniques in Robotics, Artificial Intelligence, and Kinematics, Dynamics, and Control. As the course will be project driven, prototyping skills including Matlab, Python, C, and C++ will also be important. Creative thought and enthusiasm are required.
Assessment Structure
Homework - 50%
Final project - 10% (proposal) + 25% (report) + 10% (presentation)
Participation - 5%
Student Final Projects (Selected)
Kevin Tracy.
Robust Aerocapture Guidance With a Method for Atmospheric Adaptation.
[Report]
Siddharth Gangadhar.
Safe Control for Extreme Driving with Adaptive Control Barrier Functions.
References
Optional textbook:
Goodwin. Adaptive Filtering, Prediction, and Control.
Bertsekas. Optimal Control and Reinfocement Learning.
Anderson and Moore. Optimal Control, Linear Quadratic Methods.
Borrelli. Predictive control for linear and hybrid systems.
Sutton and Barto. Reinforcement Learning: An Introduction.