INTRO TO MACHINE LEARNING

10-701, Spring 2018
GHC 4401, Mon & Wed 10:30 - 11:50 AM

Instructors Pradeep Ravikumar (pradeepr at cs dot cmu dot edu)
Manuela Veloso (mmv at cs dot cmu dot edu)
Teaching Assistants Shaojie Bai (shaojieb at andrew dot cmu dot edu
Adarsh Prasad (adarshp at andrew dot cmu dot edu)
Otilia Stretcu (ostretcu at andrew dot cmu dot edu)
Dimitris Konomis (dkonomis at andrew dot cmu dot edu)
Satyapriya Krishna (satyaprk at andrew dot cmu dot edu)
Lam Wing Chan (lamwingc at andrew dot cmu dot edu)
Wenhao Qin (wqin at andrew dot cmu dot edu)
George Stoica (gis at andrew dot cmu dot edu)
Sreena Nallamothu (snallamo at andrew dot cmu dot edu)
Office Hours Pradeep Ravikumar: GHC 8111, Mondays 1:00-2:00 PM
Manuela Veloso: TBD
Shaojie Bai: GHC 5th floor Commons, Friday 2:15pm-3:15pm
Adarsh Prasad: GHC 5th Floor Commons, Wednesday 1:45pm-2:45pm, Table 2
Otilia Stretcu: GHC 8021, Thursday 12pm - 1pm
Dimitris Konomis: Monday 7-8pm (dimitris.konomis, skype)
Satyapriya Krishna: LTI 5th Floor Kitchen, Thursday , 5-6 pm
Sreena Nallamothu: GHC 5th floor Commons, Thursday , 7-8pm
Lam Wing Chan: GHC 8th floor Kitchen Area, Tuesday 10:30am - 11:30am
George Stoica: GHC 5th floor Commons, Wednesday 4 pm - 5 pm
Wenhao Qin: GHC 5th floor Commons, Friday, 6-7pm

Grading 50% Homeworks, 25% Midterm, 25% Project

Textbooks Lectures are intended to be self-contained. For supplementary readings, with each lecture, we will have pointers to either online reference materials, or chapters from the following books:
  • CB: Pattern Recognition and Machine Learning, Christopher Bishop.
  • KM: Machine Learning: A probabilistic perspective, Kevin Murphy.
  • HTF: The Elements of Statistical Learning: Data Mining, Inference and Prediction, Trevor Hastie, Robert Tibshirani, Jerome Friedman.
  • TM: Machine Learning, Tom Mitchell.

Course Details Syllabus. Piazza Discussion Board. Homeworks. Project.

Tentative Schedule
Date Inst. Topic Readings Notes
Module: Foundations
Jan 17 MV Intro (slides) KM Chap. 1
Jan 22 PR Prob. Models: Estimators, Guarantees, MLE (slides) KM Chap. 2, 6
Jan 24 MV Prob. Models: Bayesian Estimation, MAP (slides) TM Chap. 6,
KM Chap. 5
Jan 29 PR Model-free Methods, Decision Theory (slides) HTF Chap. 2 HW1 out
Module: Prediction, Parametric Methods
Jan 31 PR Regression: Linear Regression (slides) CB Chap. 3
Feb 05 MV Regularized, Polynomial, Logistic Regression (slides) CB Chap. 3, 4
Feb 07 PR Classification: Naive Bayes, Generative vs Discriminative (slides) CB Chap. 4
Feb 12 PR Classification: Support Vector Machines (slides) KM Chap. 14 HW 1 due/
(HW2out)
Feb 14 PR Classification: Boosting, Surrogate Losses (slides) HTF Chap. 10
Feb 19 MV Decision Trees (slides) TM Chap. 3,
HTF Chap. 9
Feb 21 PR Foundations: Generalization, Model Selection (slides) HTF Chap. 7
Feb 26 MV Neural Networks and Deep Learning (slides) CB Chap. 5,
KM Chap. 28		 HW 2 due/
(HW3out)
Feb 28 MV Neural Networks and Deep Learning (slides) CB Chap. 5,
KM Chap. 28
Module: Non-Parametric Methods
March 05 PR Non-parametric Models: K nearest neighbors, kernel regression (slides) TM Chap. 8,
HTF Chap. 6, 13
Mar 07 PR Non-parametric Models: SVM, Lin Reg: primal + dual, Kernels, Kernel Trick (slides) CB Chap. 6, 7 HW 3 due (Mar 9)
Mar 12 No Class, Spring Break
Mar 14 No Class, Spring Break
Module: Unsupervised Learning
Mar 19 Midterm Review. Midterm Review (slides)
Mar 21 Midterm
Mar 26 PR Unsupervised Learning: Clustering, Kmeans (slides) HTF Chap. 14.1-14.3 (HW4out)
Mar 28 PR Unsupervised Learning: Clustering: Mixture of Gaussians, Expectation Maximization (slides) CB Chap. 9
Apr 02 PR Unsupervised Learning: Latent Variable Models (slides) CB Chap. 9
Apr 04 PR Unsupervised Learning: Graphical Models (slides) KM Chap. 10, 19, 20
Module: Sequence Models
Apr 11 MV Sequence Models: Hidden Markov Models (slides) KM Chap. 17 HW 4 due/
HW 5 out
Apr 16 MV Sequence Models: State Space Models, other time series models (slides) KM Chap. 18
Module: Representation Learning
Apr 16 TBD/PR Representation Learning: Feature Transformation, Random Features, PCA (slides) HTF Chap. 14.5
Apr 18 TBD/MV Representation Learning: PCA Contd, ICA (slides) HTF Chap. 14.7
Module: Reinforcement Learning
Apr 23 MV RL: MDPs, Value Iteration, Q Learning (slides) HW 5 due
Apr 25 MV RL: Q learning in non-det domains, Deep RL (slides)
Apr 30 PR Foundations: Statistical Guarantees for Empirical Risk Minimization (slides)
May 2 Final Project Presentations

Homeworks
  • HW 1 out. Due Monday, Feb. 12, 10:30 AM
  • HW 2 out. Due Monday, Feb. 26, 10:30 AM
  • HW 3 out. Due Friday, March 9, 10:30 AM
  • HW 4 out. Due Monday, April 9, 2018, 10:30 AM
    • Project Details