10-601 Spring 2017

Important Notes

This schedule is tentative and subject to change. Please check back often.

Lecture Videos

Panopto folder: [Andrew ID Required] https://scs.hosted.panopto.com
The above folder contains all the lecture video recordings. To access them, you will need to log in with your Andrew ID.
We have also included links to individual videos below -- however, we recommend checking the link above to find the latest videos.

Textbooks

[MC]: Machine Learning, Tom Mitchell, 1997.
[HTF]: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Trevor Hastie, Robert Tibshirani, Jerome Friedman, 2009.
[MP]: Machine Learning: A Probabilistic Perspective, Kevin R Murphy, 2012.
[BI]: Pattern Recognition and Machine Learning, Christopher Bishop, 2007.

Tentative Schedule

Date	Lecture	Readings					Announcements
		MC	HTF	MP	BI	Other
Wed, 18-Jan	Lecture 1 : Course Overview [Slides] [Video]	1	1, 2	1	1
Classification and Regression
Mon, 23-Jan	Lecture 2 : Machine Learning in Practice / k-Nearest Neighbors [Slides] [Whiteboard] [Video]	8.2	13.3	--	2.5.2
Tue, 24-Jan	Background Test (Evening)
Wed, 25-Jan	Lecture 3 : Experimental Design / k-Nearest Neighbors [Slides] [Whiteboard] [Video]	--	--	--	--		HW1 out
Mon, 30-Jan	Lecture 4 : The Probabilistic Approach to Learning from Data [Slides] [Whiteboard] [Video]	--	--	2	2	Tom Mitchell (2016, draft) Estimating Probabilities: MLE and MAP Geoff Gordon (2016) Math lecture notes from 10-600 (see Piazza note)
Wed, 1-Feb	Lecture 5 : MLE and MAP / Naive Bayes [Slides] [Whiteboard] [Video]	6.1-6.10	--	3	--	Tom Mitchell (2016, draft) Generative and Discriminative Classifiers: Naive Bayes and Logistic Regression	HW2 out HW1 due
Mon, 6-Feb	Lecture 6 : Gaussian Naive Bayes [Slides] [Whiteboard] [Video]	--	--	--	--
Wed, 8-Feb	Lecture 7 : Optimization for ML / Linear Regression [Slides] [Whiteboard] [Video]	--	--	--	--	Geoff Gordon (2016) Optimization lecture notes from 10-600 (see Piazza note) [Optional Reading] Boyd and Vandenberghe (2009) Convex Optimization (Ch. 9)
Mon, 13-Feb	Lecture 8 : Linear Regression [Slides] [Whiteboard] [Video]	--	3.1-3.4	7.1-7.3	3.1		HW3 out HW2 due
Wed, 15-Feb	Lecture 9 : Logistic Regression / Nonlinear features [Slides] [Whiteboard] [Video]	--	4.1, 4.4	8.1-8.3, 8.6	4.3.2, 4.3.4	Tom Mitchell (2016, draft) Generative and Discriminative Classifiers: Naive Bayes and Logistic Regression Charles Elkan (2014) Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training
Mon, 20-Feb	Lecture 10 : Regularization / Perceptrons and Large Margin [Slides] [Whiteboard] [Video]	4.4.0	--	8.5.4	4.1.7
Wed, 22-Feb	Lecture 11 : Kernels / Kernel Perceptron / SVMs [Slides] [Whiteboard] [Video]	--	--	14.1 - 14.2.4	6.1-6.2		HW4 out HW3 due [Course Survey due Fri, Feb 24]
Mon, 27-Feb	Lecture 12 : Kernels / SVMs [Slides] [Whiteboard] [Video]	--	12 - 12.38	14.5	7.1
Learning Theory
Wed, 1-Mar	Lecture 13 : Learning Theory (Part I) - Statistical Estimation [Slides] [Whiteboard] [Video]	7	--	--	--	Pedro Domingos (2012) A Few Useful Things to Know about Machine Learning	[HW4 due Fri, Mar 03]
Mon, 6-Mar	Lecture 14 : Midterm Exam Review [Slides] [Video]
Tue, 7-Mar	Midterm Exam (Evening Exam) 7:00pm - 9:30pm -- see Piazza for details about the location
Unsupervised Learning
Wed, 8-Mar	Lecture 15 : Clustering [Slides] [Whiteboard] [Video]	--	14.3.0	25.5	12.1, 12.3		HW5 out
Mon, 13-Mar	(No class: Midsemester break)
Wed, 15-Mar	(No class: Midsemester break)
Mon, 20-Mar	Lecture 16 : K-Means / GMMs [Slides] [Whiteboard] [Video]	6.12 - 6.12.2	8.5 - 8.5.3	11.4.1, 11.4.2, 11.4.4	9
Wed, 22-Mar	Lecture 17 : Expectation Maximization / PCA and Dimensionality Reduction [Slides] [Whiteboard] [Video]	6.12 - 6.12.2	8.5 - 8.5.3	11.4.1, 11.4.2, 11.4.4	9		HW6 out HW5 (Part I) due
Feature Learning
Mon, 27-Mar	Lecture 18 : PCA / Neural Networks [Slides] [Whiteboard] [Video]	--	14.5	12	12
Wed, 29-Mar	Lecture 19 : Neural Networks [Slides] [Whiteboard] [Video]	4	11	--	5	Michael Nielsen (2015) Neural Networks and Deep Learning (Chap. 1)
Mon, 3-Apr	Lecture 20 : Backpropagation [Slides] [Whiteboard] [Video]	--	--	--	--	Michael Nielsen (2015) Neural Networks and Deep Learning (Chap. 2) Goodfellow, Bengio, Courville (2016) Deep Learning (MLP Chapter)	HW6 due
Wed, 5-Apr	Lecture 21 : Deep Learning / CNNs [Slides] [Whiteboard] [Video]	--	--	28	--	Michael Nielsen (2015) Neural Networks and Deep Learning (Chap. 6) Goodfellow, Bengio, Courville (2016) Deep Learning (CNN Chapter)	HW7 out HW5 (Part II) due
Graphical Models
Mon, 10-Apr	Lecture 22 : Bayesian Networks (Part I) [Slides] [Whiteboard] [Video]	6.11	--	10 - 10.2.1	8.1, 8.2.2
Wed, 12-Apr	Lecture 23 : Bayesian Networks (Part II) [Slides] [Whiteboard] [Video]	6.11	--	10 - 10.2.1	8.1, 8.2.2
Mon, 17-Apr	Lecture 24 : Hidden Markov Models [Slides] [Whiteboard] [Video]	--	--	10.2.2 - 10.2.3	13.1-13.2	[Only pages 257 - 266] Rabiner (1989) A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition	HW8 out HW7 due
Learning Paradigms
Wed, 19-Apr	Lecture 25 : Matrix Factorization and collaborative filtering [Slides] [Whiteboard] [Video]	--	--	--	--	Koren, Bell, and Volinsky (2009) Matrix Factorization Techniques for Recommender Systems
Mon, 24-Apr	Lecture 26 : Reinforcement Learning [Slides] [Video]	13	--	--	--	Kaelbling, et al (1996) Reinforcement Learning: A Survey	HW9 out HW8 due
Wed, 26-Apr	Lecture 27 : Information Theory [Slides] [Video]	7	--	--	--
Learning Theory
Mon, 1-May	Lecture 28 : Learning Theory (Part II) - PAC Learning [Slides] [Whiteboard] [Video]	7	--	--	--
Wed, 3-May	Lecture 29 : Final Exam Review [Slides] [Whiteboard] [Video]						HW9 due
Mon, 8-May	Final exam, 5:30pm - 08:30pm

Introduction to Machine Learning

Important Notes

Lecture Videos

Textbooks

Tentative Schedule

Classification and Regression

Learning Theory

Unsupervised Learning

Feature Learning

Graphical Models

Learning Paradigms

Learning Theory