Advanced Machine Learning: Theory and Methods

10-716, Spring 2021
Zoom, Tue & Thurs 1:30PM - 2:50PM

Instructor Pradeep Ravikumar (pradeepr at cs dot cmu dot edu)
Teaching Assistants Yusha Liu (yushal at andrew dot cmu dot edu)
Quang Minh Hoang (qhoang at andrew dot cmu dot edu)
Ojash Neopane (oneopane at andrew dot cmu dot edu)
Chang Shi (changshi at andrew dot cmu dot edu)
Office Hours Pradeep Ravikumar: Tuesdays & Thursdays: 3:40pm - 4:00pm
The office hours of the TAs will be posted on Piazza.

Key Course Links Syllabus. Piazza. Gradescope. Review Material.

Discussion, Announcements We will use Piazza for announcements, providing resource materials, as well as the discussion board for the class.

Textbooks Lecture notes will be posted for each class, which will be largely self-contained.

Schedule
Date Topic Readings Note
02-Feb Decision Theory: Uniform Optimality, Empirical Loss Principle
04-Feb Decision Theory: Perils of Likelihood Principle
09-Feb Non-parametric Bayes
11-Feb Non-parametric Bayes HW1 out
16-Feb Non-parametric Density Estimation
18-Feb Non-parametric Density Estimation
23-Feb No Class: Break day
25-Feb Non-parametric Regression HW1 due; HW2 out
2-Mar Non-parametric Regression
4-Mar Non-parametric Classification
9-Mar Linear/Additive Non-parametric Estimation
11-Mar Deep Density Estimation HW2 due; HW3 out
16-Mar Deep Density Estimation
18-Mar Optimal Transport
23-Mar Optimal Transport
25-Mar High-Dimensional Regression HW3 due; HW4 out
30-Mar High-Dimensional Regression Project Proposal due
01-Apr Deep Neural Networks and Kernels
06-Apr Deep Neural Networks and Kernels
08-Apr Dimensionality Reduction/Manifolds HW4 due; HW5 out
13-Apr Dimensionality Reduction/Manifolds
14-Apr Project Milestone due
15-Apr No Class: Spring Carnival
20-Apr Clustering
22-Apr Clustering HW5 due
27-Apr Learning and Games
29-Apr Random Forests
04-May Project Spotlights
06-May Project Spotlights
11-May Final Project Report Due

Homeworks There will be five homework assignments, approximately evenly spaced throughout the semester. The assignments will be posted on the course website, and on Piazza. We will use Gradescope for submitting, and grading assignments. You will get a late day quota of 10 days, which you can distribute among the five homeworks as you wish, subject to a maximum of 3 days per homework. Homeworks submitted after your late day quota will lose all points. The homework schedule is posted right at the beginning of the semester, so please plan in advance. We expect you to use the late day quota for conference deadlines and events of the like, so we cannot provide an additional extension for such cases. In the case of an emergency (sudden sickness, family problems, etc.), we can give you a reasonable extension. But we emphasize that this is reserved for true emergencies.

Class Project There will be a class project. You can form groups of up to 3 students.

Grading 75% Homeworks, 25% Project