16-633: Robot Cognition and Learning
Spring 2024
Instructor: Chris Atkeson,
cga@cmu.edu
TA: Viekash Vinoth Kumar, vvinothk@andrew.cmu.edu
Class time: MW 3:30-4:20PM
Place: WEH 2302
Units: 12
For both undergraduate and graduate students
Schedule
-
Jan 17:
Introduction:
Go over class web page
Introduction to the course
-
Jan 22:
Object detection.
notes.txt
Paper: Object Detection in 20 Years: A Survey
-
Jan 24:
Object detection continued.
-
Jan 29:
Video analysis: Using self supervision (bottom-up processing) to learn
appearance and shape models.
-
Jan 31:
What can we do with object detection?
-
Feb 5:
Not making the same mistake twice.
-
Feb 7:
Discussing class robot.
-
Feb 12:
What should the units of behavior be?
Slides
-
Feb 14:
Using LLMs
-
To be scheduled.
-
April 22: Project presentations
-
April 24: Project presentations
-
May 6 - Graduating students have to have turned everything in.
-
May 12 - All students have to have turned everything in.
Assignments
-
Assignment 0: Due Jan 20.
Send email to Chris and cc the TA:
Who are you? Why are you here?
Do you have any relevant experience or prior work?
Any ideas about the project you want to do? ("I don't know what project
I want to do." is okay.)
What topics would you like the course to cover?
Google and send us some interesting URLs.
Are there other versions of this course at other places
or useful web pages
we should look at (send URLs)?
Be sure your name is obvious in the email, and you mention the course
name or number in the subject line.
I teach more than one course, and a random email from
robotlover@cs.cmu.edu is hard for us to process.
-
Assignment 1: Due Jan 28.
Get some kind of object detector running on your laptop or desktop computer.
A popular choice is something from the YOLO series (another writeup) (and another writeup). Detectron2 is another popular object detector that
also does segmentation (creates masks). CenterNet avoids the use of region proposals.
Ways to find others include googling
"object detector" or "object detector tutorial",
Papers With Code,
and searching on github: object detection and real time object detection. The software should name the object,
and locate it in the image using a bounding box or a mask.
Try using the object detector for tracking, using naming to help with
associating objects to tracks, and the bounding box or mask centroid
and some measure of angle to provide the object pose in each image.
Project
Each group will have their own robot to work with.
However, I am very interested multi-robot tasks, so it would be
great if groups can sometimes work together.
We will start with
this robot,
and escalate if that is useful.
We will work out the specific project topics together.
The ideal project would involve creating a system that we could
deploy in Newell Simon Hall (kiosk or roving robot).
See the deadlines in the schedule (above).
You can work in groups or alone.
The "deliverables" include
a github web page
providing your code and
explaining how others could do your project and improve on your results.
You will also present your project, and ideally the presentation should
be made public as part of your web page. There will be intermediate
deliverables including draft web pages and practice presentations.
More Resources
-
Github
-
Hugging Face
-
Papers with code
Course poster, suitable from framing:
