16-299: Introduction to Feedback Control Systems
Spring 2025
Instructor: Chris Atkeson, cga@cmu.edu, Youtube channel
TA: Krishna Suresh, ksuresh2@andrew.cmu.edus, Introducing myself
TA Office hours: TBA
Class time: MW 11-12:20PM
Place: WEH 5415
Units: 12
Piazza

Sign up for Piazza

We are revising this course to better fit the new Robotics Major. The new title and course description will be. Making Decisions: Robot Control, Planning, and Learning

This undergraduate robotics course covers the fundamental concepts of feedback control and planning algorithms for robotics. The course presents a unified approach to feedback control/policy design, optimal control, planning, and reinforcement learning. Topics include robot dynamics, PID control, LQR control, state estimation including Kalman filters, insight from frequency domain techniques, path planning algorithms such as A* and Rapidly-Exploring Random Trees (RRT), dynamic programming, and reinforcement learning. Students will gain practical experience in designing and implementing feedback control and planning both in simulation and on a real system. The course includes lectures, assignments, work with a real robot, and a project. By the end of the course, students will have a strong foundation in feedback control and planning algorithms for robotics, including reinforcement learning, and have gained insight into how to make decisions and choose actions.

Last year's course

Course administration and policies

Events

Videos

• Starship 5 booster catch, Another view, showing thrust vectoring, closeup
• Starship 6 landing (in ocean)
• Falcon 9 landings,
• Falcon Heavy
• Does this make a good project?
• SpaceX fails
• Chinese test flight
• Goodby HD Atlas
• Electric Atlas autonomous
• Atlas, Behind the scenes,
• Older Atlas video, Older Behind the scenes, Writeup
• Robot learning pole balancing, Robot learning swingup

Of Interest

Textbook

There is a second book I urge you to read and the PDF is available from the CMU library: Feedback Control for Everyone

Schedule

• Jan 13:
  Atkeson introduction. Atkeson Youtube channel.
  Go over class web page
  Introduction to the course:
  How would you program the SpaceX rockets?
  Intuition and a useful mental model: Springs and dampers, natural frequency, damping ratio
  Examples: Thermostat, body temperature, breathing, blood pressure, gaze, driving (direction, cruise control), rockets (including landing), game-playing (alpha-zero), ...
  Assignment 0
  Assignment 1
  CGA Lecture 1
  Read Textbook: Chapter 1 - Introduction
  
• Jan 15:
  Simulation
  Matlab example.
  Genesis simulator
  
• Jan 22:
  Models, states, actions/controls,
  continuous and discrete time dynamics functions,
  What do you want? Goals/optimization_functions/rewards, and control laws/policies.
  CGA Lecture 2
  Jan 27:
  What is a plan?
  Indexing on state or time (you can include time as an element of the state).
  Compare to using state feedback to implement a plan.
  Open loop whipping
  Feedforward control.
  Swingup
  When do we need feedback?
  Paddle juggling w/ vision, Paddle juggling w/ vision 10 years later, Rizzi paddle juggling, 3 ball blind paddle juggling
  
• Jan 29: The state space approach:
  CGA Lecture 4
  Read Textbook: Chapter 6 - Linear Systems
  
• Feb 3: Build our lab robots. Bring laptop. Try to install Arduino IDE in advance.
• Feb 5: TWIP modeling: CGA Lecture 10
• Feb ?: More feedback and feedforward control
  Integral control.
  Learning feedforward control from practice.
  CGA Lecture 7
  
• Feb ?: Optimization:
  kinematics example.
  Redundant inverse kinematics:
  Unconstrained optimization: Using Matlab's fminsearch and fminunc.
  Unconstrained optimization with soft constraints: Using Matlab's fminsearch and fminunc, with desired posture.
  Constrained optimization: Using Matlab's fmincon.
  CGA Lecture 8
  
• Feb ?: Optimization continued. Learning kinematics.
  
• Feb ?: Unifying ideas: enumerating and evaluating options, forward search, how computers play games, value/heuristic functions, dynamic programming.
• Feb ?: Linear State Feedback: Eigenvalues;
  CGA Lecture 5
  Read Textbook: Chapter 7 - State Feedback
  
• Feb ?: Linear Quadratic Regulators (LQR)
  CGA Lecture 5
  Read Textbook: Chapter 7 - State Feedback
  
• Feb ?: State-estimation, Linear State Observers: Observability; Observer derivation and design.
  CGA Lecture 6
  Read Textbook: Chapter 8 - Output Feedback
  
• Feb ?:
• March ?: Thinking in the frequency domain,
  CGA Lecture 9
  
• March ?: CGA Lecture 7
  CGA Lecture 8
  CGA Lecture 12
  Dynamic Programming
  
• To be scheduled.
• April 21: Project presentations
• April 23: Project presentations
• May 5 - Graduating students have to have turned everything in.
• May 11 - All students have to have turned everything in.

Topics

• UNDER CONSTRUCTION
  What you see are previous year's topics.
  
  
• Unit 0: Intuition, PID
• Unit I: State Space
  • Plans and feedforward control, clocks, phases, and internal states.
  • Output Feedback
    Read Textbook: Chapter 8 - Output Feedback
    
  • Learning: Optimization, Optimal control, and Reinforcement learning.
    
• Unit II: Frequency Domain Previous year's notes
  • Read Textbook: Chapter 2 - Feedback Principles
    
  • Mathematical models: ODEs, diff. equations, and transfer functions. Previous year's notes.
  • Conceptual overview, complex variables
  • Transfer Functions: frequency response, linearity
    Read Textbook: Chapter 9 - Transfer Functions
    
  • Complex Plane Intuition: 2nd order systems, moving poles with P and D
    Read Textbook: Chapter 10 - Frequency Domain Analysis
    
  • Frequency Domain Design
    Read Textbook: Chapter 12 - Frequency Domain Design
    
  • Integral Control: steady state error rejection, benefits of added state
• Unit III: Nonlinear Systems
  Nonlinear EOMs, eq. points, and linearization.
  • Revisit ODE models, attractors (equilibrium points), saddle points, repellers, and stability;
  • Dynamic programming
  • Nonlinear Stability: Lyapunov methods: introduction and simple cases.
    Read Textbook: Chapter 5 - Dynamic Behavior
    Previous year's notes
  • Trajectory Following: pure pursuit, tubes: linearization about a trajectory;
• Unit IV: Additional topics: class suggests more and decides which to talk about.
  • System identification
  • Adaptive control
  • Optimal control and reinforcement learning, dynamic programming
  • Robust control
    Read Textbook: Chapter 13 - Robust Performance
    
  • Fundamental limits
    Read Textbook: Chapter 14 - Fundamental Limits
    
  • Architecture (top-down, bottom-up), system, and control design, interaction
    Adaptation, Learning and Cognition
    Textbook: Chapter 15 - Architecture and System Design
    
  • Hybrid systems (Discrete and continuous states)
• Case studies: class suggests more and decides which to talk about.
  • Textbook: Chapter 3, 4, and 15
    
  • scribbled models from previous years
  • Inverted pendulums: the ballbot, rocket control and landing
  • Vehicle Modeling: kinematic models, nonholonomic constraints;
  • UAVs - equations of motion, decomposition, differential flatness, architecture, trajectory planning, controller design
  • Running and walking

Assignments

• Assignment 0: Due Jan 25. Send email to Chris and the TA: Who are you? Why are you here? Done any control (in simulation or on physical machines)? Any ideas about the project you want to do? ("I don't know what project I want to do." is okay.) What topics would you like the course to cover? Google and send us some interesting URLs. What example of control really impressed you (Boston Dynamics Atlas jumping around, SpaceX landing rockets on a barge, ...; send URLs of web pages or videos)? Are there other versions of this course at other places or useful web pages we should look at (send URLs)?
  Be sure your name is obvious in the email, and you mention the course name or number in the subject line. I teach more than one course, and a random email from robotlover@cs.cmu.edu is hard for us to process.
• Assignment 1: Due Jan 21. Get access to MATLAB. Download these Matlab examples. Play with the example mass-kb-manual by changing the stiffness and damping. Nothing to turn in.
• Assignment 2: Due Jan 28. Assignment 2
• More later.

Project

We will work out the project topics together. The ideal project would involve modeling and controlling a real physical system. It would be cool to have a learning component. Second best is doing everything in simulation. We can help you buy a real physical system, if we don't have what you want already. We can help you get access to robots.

See the deadlines in the schedule (above). You can work in groups or alone. The "deliverables" include a web page along the lines of an instructables web page explaining how others could do your project and improve on your results. You will also present your project, and ideally the presentation should be made public as part of your web page. There will be intermediate deliverables including draft web pages and practice presentations.

More Resources

• Wikipedia
• Introduction to feedback control systems: course notes