16-883 Provably safe robotics (Spring 2021)
(Last Update: 1/19/2020)
Time: Tuesday and Thursday 8:20-9:40am
Location: zoom
Instructor: Changliu Liu, cliu6@andrew.cmu.edu
Office hours: by appointment
Canvas: https://canvas.cmu.edu/courses/21753
Piazza: https://piazza.com/class/kk4dtp79ysa5kv
Course Description
Safe autonomy has become increasingly critical in many application domains. It is important to ensure not only the safety of the ego robot, but also the safety of other agents (humans or robots) that directly interact with the autonomy. For example, robots should be safe to human workers in human-robot collaborative assembly; autonomous vehicles should be safe to other road participants. For complex autonomous systems with many degrees of freedom, safe operation depends on the correct functioning of all system components, i.e., accurate perception, optimal decision making, and safe control.
This course deals with both the design and the verification of safe robotic systems. From the design perspective, we will talk about how to assure safety through planning, prediction, learning, and control. From the verification perspective, we will talk about verification of deep neural networks, safety or reachability analysis for closed loop systems, and analysis of multi-agent systems.
Key Topics
safe control
safe learning
reachability analysis
verification of deep learning systems
Course Goal
- To get familiar with tools and methodologies available to help the design and verification of safe robotic systems.
- To use the tools and methodologies to improve the safety level of existing designs or verify existing designs.
- To understand how to develop new tools and methodologies for robot safety.
Syllabus
Prerequisite
Linear algebra, programming, robot kinematics (recommended not required)
Assessment Structure
In-class paper presentation - 10%
Project proposal - 10%
Midterm project check-in -10%
Final project report - 20%
Final project presentation - 10%
Notes and summary - 20%
Participation - 20%
Expectation
This will be a seminar-style course with 35% lectures, 35% guest lectures, 30% student paper presentation.
Extra time commitment is around 5h per week.
The assessment is leaned toward the course project.
Some suggested topics for the final project are listed below:
1. Apply some design method to your own research problem to improve the safety level of your system.
2. Apply some verification method to your own research problem to demonstrate the safety of your system.
3. Apply some design or verification method on safe human-robot interactions.
4. Apply some design or verification method on safe autonomous driving.
5. Develop new algorithms for safe control, safe planning, safe prediction, etc.
6. Develop new algorithms for reachability analysis, robustness analysis, etc.
7. Develop new algorithms for verifying deep learning systems, deep neural networks, etc.
8. Develop new algorithms for synthesizing safe multi-agent systems.
Student Projects (Selected)
Jack Good. Tree-Based Learned Models for Safe Robot Control [Published Version]
Ruixuan Liu. IADA: Iterative Adversarial Data Augmentation using Verification
Tianhao Wei. Safe Control with Neural Network Dynamic Models [Published Version]
References
Literature:
A survey of methods for safe human-robot interaction
Algorithms for verifying deep neural networks [NeuralVerification.jl]
Related course:
Safe and interactive robotics (Stanford)