Advanced Machine Learning: Theory and Methods

10-716, Spring 2020
WH 7500, Tue & Thurs 1:30PM - 2:50PM

Instructor Pradeep Ravikumar (pradeepr at cs dot cmu dot edu)
Teaching Assistants Ian Char (ichar at cs dot cmu dot edu)
Kartik Gupta (kartikg1 at andrew dot cmu dot edu)
Tom Yan (tyan2 at andrew dot cmu dot edu)
Sean Jin (seanj at andrew dot cmu dot edu)
Office Hours Pradeep Ravikumar: Thursday: 3:00pm - 4:00pm in GHC 8111
The office hours of the TAs will be posted on Piazza.

Course details Syllabus. Piazza. Homeworks. Review Material.

Discussion, Announcements We will use Piazza for announcements, providing resource materials, as well as the discussion board for the class.

Textbooks Lecture notes will be posted for each class, which will be largely self-contained.

Tentative Schedule
Date Topic Readings Notes
Jan 14 Decision Theory: Empirical Loss Principle Lecture Notes
Jan 16 Decision Theory: Perils of Likelihood Principle Lecture Notes
Jan 21 Non-parametric Bayes Lecture Notes
Jan 23 Non-parametric Bayes (contd.)
Jan 28 Non-parametric Density Estimation Lecture Notes
Jan 30 Non-parametric Density Estimation (contd.) HW1 out
Feb 4 Non-parametric Regression Lecture Notes
Feb 6 Non-parametric Regression (contd.)
Feb 11 Nonparametric-Regression (contd.)
Feb 13 Non-parametric Classification Lecture Notes HW1 due
Feb 18 Linear/Additive Non-parametric Estimation Lecture Notes
Feb 20 Midterm
Feb 25 Deep Density Estimation Lecture Notes HW2 out; Project Proposal Due
Feb 27 Deep Density Estimation (contd.)
Mar 3 Optimal Transport Lecture Notes
Mar 5 Optimal Transport (contd.) HW2 due
Mar 10 No Class: Spring Break
Mar 12 No Class: Spring Break
Mar 17 No Class
Mar 19 High-Dimensional Regression Lecture Notes
Mar 24 High-Dimensional Regression (contd.) HW3 out
Mar 26 Deep Neural Networks and Kernels Lecture Notes
Mar 31 Deep Neural Networks and Kernels (contd.)
Apr 2 Dimensionality Reduction/Manifolds Lecture Notes
Apr 7 Dimensionality Reduction/Manifolds (contd.) HW3 due
Apr 9 Clustering Lecture Notes HW4 out; Project Milestone Report Due
Apr 14 Clustering (contd.)
Apr 16 Clustering (contd.)
Apr 21 Random Forests Lecture Notes
Apr 23 Causality Lecture Notes HW 4 due
Apr 28 Project Spotlights Presentations
Apr 30 Project Spotlights Presentations
May 7 Final Project Report Due

Homeworks There will be four homework assignments, approximately evenly spaced throughout the semester. The assignments will be posted on the course website, and on Piazza. We will use Gradescope for submitting, and grading assignments. You will get a late day quota of 8 days, which you can distribute among the four homeworks as you wish, subject to a maximum of 3 days per homework. Homeworks submitted after your late day quota will lose all points. The homework schedule is posted right at the beginning of the semester, so please plan in advance. We expect you to use the late day quota for conference deadlines and events of the like, so we cannot provide an additional extension for such cases. In the case of an emergency (sudden sickness, family problems, etc.), we can give you a reasonable extension. But we emphasize that this is reserved for true emergencies.

Exam There will be one midterm exam, scheduled halfway through. The exam will consist of multiple choice and true/false questions, as well as short answer questions, and will be open book and open notes.

Class Project There will be a class project. You can form groups of up to 2 students. Further details can be found here.

Grading 50% Homeworks, 25% Midterm Exam, 25% Project