16-299: Introduction to Feedback Control Systems
Spring 2024
Instructor: Chris Atkeson, cga@cmu.edu
TA: Sparsh Garg, sparshg@andrew.cmu.edu
TA Office hours: Friday 2:30-3:30
Class time: MW 11-12:20PM
Place: WEH 5415
Units: 12

Last year's course

Course administration and policies

Events

Video of the day

• SpaceX landings, Feedback control is cool Does this make a good project?
• Atlas, Behind the scenes,
• Older Atlas video, Older Behind the scenes, Writeup
• Robot learning pole balancing, Robot learning swingup

Of Interest

Textbook

There is a second book I urge you to read and the PDF is available from the CMU library: Feedback Control for Everyone

Schedule

• Jan 17:
  Atkeson introduction. Atkeson Youtube channel.
  Go over class web page
  Introduction to the course:
  How would you program the SpaceX rockets?
  Intuition and a useful mental model: Springs and dampers, natural frequency, damping ratio
  Examples: Thermostat, body temperature, breathing, blood pressure, gaze, driving (direction, cruise control), rockets (including landing), game-playing (alpha-zero), ...
  Assignment 0
  Assignment 1
  CGA Lecture 1
  Read Textbook: Chapter 1 - Introduction
  
• Jan 22:
  Matlab example.
  

  Do m*xdd + k*x = 0 on board
  sin(wt) and cos(wt) satisfy this equation with w=sqrt(k/m)
  Then talk about m*xdd + b*xd + k*x = 0
  x = e^(st)
  dx/dt = e^(st)*s
  m*e^(st)*s^2 + b*e^(st)*s + k*e^(st) = 0
  m*s^2 + b*s + k = 0
  Apply the quadratic formula:
  s = -b/(2*m) +/- i*sqrt( b^2 - 4*m*k )
  aside: e^(n + i*p) = (e^n)(cos(p) + i*(sin(p)))
  --- engineers use j for i, since i is already used for electrical current
  

  This is prettier if we invent a new term: damping_ratio = b/(2*sqrt(km))
  s^2 + 2*w*damping_ratio*s + w^2 = 0
  s = -w( damping_ratio +/- i*sqrt( 1 - damping_ratio^2 )
  damping_ratio = b/(2*sqrt(km)) is known as the damping ratio.
  damping_ratio < 1 -> oscillation
  damping_ratio >= 1 -> no oscillation
  

  Finish CGA Lecture 1
  

• Jan 24: Models, states, actions/controls,
  continuous and discrete time dynamics functions,
  What do you want? Goals/optimization_functions/rewards, and control laws/policies.
  CGA Lecture 2
  
• Jan 29: What is a plan?
  Open loop whipping
  Feedforward control.
  Swingup
  When do we need feedback?
  Paddle juggling w/ vision, Paddle juggling w/ vision 10 years later, Rizzi paddle juggling, 3 ball blind paddle juggling
  Compare to using state feedback to implement a plan.
  Integral control.
  Learning feedforward control from practice.
  CGA Lecture 7
  
• Jan 31: Optimization:
  kinematics example.
  Redundant inverse kinematics:
  Unconstrained optimization: Using Matlab's fminsearch and fminunc.
  Unconstrained optimization with soft constraints: Using Matlab's fminsearch and fminunc, with desired posture.
  Constrained optimization: Using Matlab's fmincon.
  CGA Lecture 8
  
• Feb 5: Optimization continued. Learning kinematics.
  
• Feb 7: The state space approach:
  CGA Lecture 4
  Read Textbook: Chapter 6 - Linear Systems
  
• Feb 12: Build our lab robots.
• Feb 14: (Happy Valentines Day!) Linear State Feedback: Eigenvalues;
  CGA Lecture 5
  Read Textbook: Chapter 7 - State Feedback
  
• Feb 19: Linear Quadratic Regulators (LQR)
  CGA Lecture 5
  Read Textbook: Chapter 7 - State Feedback
  
• Feb 21: State-estimation, Linear State Observers: Observability; Observer derivation and design.
  CGA Lecture 6
  Read Textbook: Chapter 8 - Output Feedback
  
• Feb 26: TWIP modeling: CGA Lecture 10
• Feb 28:
• March 11: Thinking in the frequency domain,
  CGA Lecture 9
  
• March 13: CGA Lecture 7
  CGA Lecture 8
  CGA Lecture 12
  Dynamic Programming
  
• To be scheduled.
• April 22: Project presentations
• April 24: Project presentations
• May 6 - Graduating students have to have turned everything in.
• May 12 - All students have to have turned everything in.

Topics

• UNDER CONSTRUCTION
  What you see are previous year's topics.
  
  
• Unit 0: Intuition
• Unit I: State Space
  • Plans and feedforward control, clocks, phases, and internal states.
  • Output Feedback
    Read Textbook: Chapter 8 - Output Feedback
    
  • Learning: Optimization, Optimal control, and Reinforcement learning.
    
• Unit II: Frequency Domain Previous year's notes
  • Read Textbook: Chapter 2 - Feedback Principles
    
  • Mathematical models: ODEs, diff. equations, and transfer functions. Previous year's notes.
  • Conceptual overview, complex variables
  • Transfer Functions: Laplace transforms, frequency response, linearity
    Read Textbook: Chapter 9 - Transfer Functions
    
  • Complex Plane Intuition: 2nd order systems, moving poles with P and D
    Read Textbook: Chapter 10 - Frequency Domain Analysis
    
  • Frequency Domain Design
    Read Textbook: Chapter 12 - Frequency Domain Design
    
  • Integral Control: steady state error rejection, benefits of added state
• Unit III: Nonlinear Systems
  Nonlinear EOMs, eq. points, and linearization.
  • Revisit ODE models, attractors (equilibrium points), saddle points, repellers, and stability;
  • Dynamic programming
  • Nonlinear Stability: Lyapunov methods: introduction and simple cases.
    Read Textbook: Chapter 5 - Dynamic Behavior
    Previous year's notes
  • Trajectory Following: pure pursuit, tubes: linearization about a trajectory;
• Unit IV: Additional topics: class suggests more and decides which to talk about.
  • System identification
  • Adaptive control
  • Optimal control and reinforcement learning, dynamic programming
  • Robust control
    Read Textbook: Chapter 13 - Robust Performance
    
  • Fundamental limits
    Read Textbook: Chapter 14 - Fundamental Limits
    
  • Architecture (top-down, bottom-up), system, and control design, interaction
    Adaptation, Learning and Cognition
    Textbook: Chapter 15 - Architecture and System Design
    
  • Hybrid systems (Discrete and continuous states)
• Case studies: class suggests more and decides which to talk about.
  • Textbook: Chapter 3, 4, and 15
    
  • scribbled models from previous years
  • Inverted pendulums: the ballbot, rocket control and landing
  • Vehicle Modeling: kinematic models, nonholonomic constraints;
  • UAVs - equations of motion, decomposition, differential flatness, architecture, trajectory planning, controller design
  • Running and walking

Assignments

• Assignment 0: Due Jan 25. Send email to Chris and the TA: Who are you? Why are you here? Done any control (in simulation or on physical machines)? Any ideas about the project you want to do? ("I don't know what project I want to do." is okay.) What topics would you like the course to cover? Google and send us some interesting URLs. What example of control really impressed you (Boston Dynamics Atlas jumping around, SpaceX landing rockets on a barge, ...; send URLs of web pages or videos)? Are there other versions of this course at other places or useful web pages we should look at (send URLs)?
  Be sure your name is obvious in the email, and you mention the course name or number in the subject line. I teach more than one course, and a random email from robotlover@cs.cmu.edu is hard for us to process.
• Assignment 1: Due Jan 21. Get access to MATLAB. Download these Matlab examples. Play with the example mass-kb-manual by changing the stiffness and damping. Nothing to turn in.
• Assignment 2: Due Jan 28. Assignment 2
• Assignment 3: Due Feb 11. Assignment 3, matlab-code-for-ass3.zip for the example Matlab code.
• Assignment 4: Due Feb 18. Assignment 4, Nothing to turn in.
• Assignment 5: Due Mar 10. (You can turn it in earlier and have a better vacation). Assignment 5
• Assignment 6: Due Mar 31. Assignment 6

Project

We will work out the project topics together. The ideal project would involve modeling and controlling a real physical system. It would be cool to have a learning component. Second best is doing everything in simulation. We can help you buy a real physical system, if we don't have what you want already. We can help you get access to robots.

See the deadlines in the schedule (above). You can work in groups or alone. The "deliverables" include a web page along the lines of an instructables web page explaining how others could do your project and improve on your results. You will also present your project, and ideally the presentation should be made public as part of your web page. There will be intermediate deliverables including draft web pages and practice presentations.

More Resources

• Wikipedia
• Introduction to feedback control systems: course notes