Introduction to Machine Learning

Important Notes

This schedule is tentative and subject to change. Please check back often.

Tentative Schedule

Date	Lecture	Readings	Announcements
Classification & Regression
Mon, 26-Aug	Lecture 1 : Course Overview [Slides] [Slides (Inked)]	Command Line and File I/O Tutorial. 10601 Course Staff (2020). 10601 Learning Objectives. Matt Gormley (2023). Math Resources. 10601 Course Staff (2023).	HW1 Out
Wed, 28-Aug	Lecture 2 : Machine Learning as Function Approximation [Slides] [Slides (Inked)]	10601 Notation Crib Sheet. Matt Gormley (2023).
Fri, 30-Aug	Recitation: HW1 [Handout] [Solutions]
Mon, 2-Sep	(Labor Day - No Class)
Wed, 4-Sep	Lecture 3 : Decision Trees [Slides] [Slides (Inked)] [Poll]	Visual Information Theory. Christopher Olah (2015). blog. Decision Trees. Hal Daumé III (2017). CIML, Chapter 1.	HW1 Due HW2 Out
Fri, 6-Sep	Recitation: HW2 [Handout] [Solutions]
Mon, 9-Sep	Lecture 4 : k-Nearest Neighbors [Slides] [Slides (Inked)] [Poll]	Geometry and Nearest Neighbors. Hal Daumé III (2017). CIML, Chapter 3.
Wed, 11-Sep	Lecture 5 : Model Selection and Experimental Design [Slides] [Slides (Inked)] [Poll]	Limits of Learning. Hal Daumé III (2017). CIML, Chapter 2.
Linear Models
Fri, 13-Sep	Lecture 6 : Perceptron [Slides] [Slides (Inked)] [Poll]	The Perceptron. Hal Daumé III (2017). CIML, Chapter 4.
Mon, 16-Sep	Lecture 7 : Linear Regression [Slides] [Slides (Inked)] [Poll]	Linear Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 7.1-7.3.	HW2 Due HW3 Out
Wed, 18-Sep	Lecture 8 : Optimization for ML [Slides] [Slides (Inked)] [Poll]	An overview of gradient descent optimization algorithms. Sebastian Ruder (2017).
Fri, 20-Sep	Recitation: HW3 [Handout] [Solutions]		Exam 1 Practice Problems out
Mon, 23-Sep	Lecture 9 : Stochastic Gradient Descent / Logistic Regression [Slides] [Slides (Inked)] [Poll]	Estimating Probabilities: MLE and MAP. Tom Mitchell (2016). Probability and Information Theory. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 3. Logistic Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 1.4.6, 8.1-8.3, 8.6.	HW3 Due (only two grace/late days permitted)
Wed, 25-Sep	Lecture 10 : Feature Engineering / Regularization [Slides] [Slides (Inked)] [Poll]	Regularization for Deep Learning. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 7.1 and 7.8.
Fri, 27-Sep	Exam 1 Review OH
Neural Networks
Mon, 30-Sep	Lecture 11 : Neural Networks [Slides] [Slides (Inked)] [Poll]	Deep Feedforward Networks. Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016). Deep Learning, Chapter 6.1-6.4.
Mon, 30-Sep	Exam 1 (evening exam, details will be announced on Piazza) 6:30p - 8:30p		HW4 Out
Wed, 2-Oct	Lecture 12 : Backpropagation I [Slides] [Slides (Inked)] [Poll]
Fri, 4-Oct	Recitation: HW4 [Handout] [Solutions]
Mon, 7-Oct	Lecture 13 : Backpropagation II [Slides] [Slides (Inked)] [Poll]
Societal Impacts
Wed, 9-Oct	Lecture 14 : Societal Impacts of ML [Slides] [Slides (Inked)] [Poll]		HW4 Due HW5 Out
Fri, 11-Oct	Recitation: HW5 [Handout] [Solutions]
Mon, 14-Oct	(Fall Break - No Class)
Tue, 15-Oct
Wed, 16-Oct	(Fall Break - No Class)
Thu, 17-Oct
Fri, 18-Oct	(Fall Break - No Class)
Learning Theory
Mon, 21-Oct	Lecture 15 : PAC Learning [Slides] [Slides (Inked)] [Poll]
Wed, 23-Oct	Lecture 16 : PAC Learning / MLE & MAP [Slides] [Slides (Inked)] [Poll]
Fri, 25-Oct	Recitation: HW6 [Handout] [Solutions]
Sun, 27-Oct		Deep learning. Yann LeCun, Yoshua Bengio, & Geoffrey Hinton (2015). Nature.	HW5 Due HW6 Out, Exam 2 Practice Problems out
Deep Learning
Mon, 28-Oct	Lecture 17 : CNNs and RNNs [Slides] [Slides (Inked)] [Poll]
Wed, 30-Oct	Lecture 18 : RNN-LMs and Transformers-LMs [Slides] [Slides (Inked)] [Poll]
Fri, 1-Nov	Exam 2 Review OH
Sat, 2-Nov			HW6 Due (only two grace/late days permitted)
Mon, 4-Nov	Lecture 19 : AutoDiff, Pre-training, Fine-Tuning, In-context Learning [Slides] [Slides (Inked)] [Poll]
Reinforcement Learning
Wed, 6-Nov	Lecture 20 : Reinforcement Learning: MDPs [Slides] [Slides (Inked)] [Poll]
Thu, 7-Nov	Exam 2 (evening exam, details will be announced on Piazza) 6:45p - 8:45p		HW7 Out
Fri, 8-Nov	Recitation: HW7 [Handout] [Solutions]
Mon, 11-Nov	Lecture 21 : Reinforcement Learning: Value/Policy Iteration [Slides] [Slides (Inked)] [Poll]
Wed, 13-Nov	Lecture 22 : Reinforcement Learning: Q-Learning / Deep RL [Slides] [Slides (Inked)] [Poll]
Fri, 15-Nov	Recitation: HW8 [Handout] [Solutions]
Sun, 17-Nov		A Tutorial on Principal Component Analysis. Jonathon Shlens (2014).	HW7 Due HW8 Out
Learning Paradigms
Mon, 18-Nov	Lecture 23 : Recommender Systems / Ensemble Methods: Boosting [Slides] [Slides (Inked)] [Poll]
Wed, 20-Nov	Lecture 24 : Ensemble Methods: Bagging / K-Means [Slides] [Poll]
Fri, 22-Nov	No Recitation
Mon, 25-Nov	Lecture 25 : Dimensionality Reduction: PCA [Poll]		HW8 Due HW9 Out
Wed, 27-Nov	(Thanksgiving Break - No Class)
Thu, 28-Nov
Fri, 29-Nov	(Thanksgiving Break - No Class)
Mon, 2-Dec	Recitation: HW9		Exam 3 Practice Problems out
Wed, 4-Dec	Lecture 26 : Special Topics: Generative Models for Vision / Significance Testing for ML [Poll]
Thu, 5-Dec			HW9 Due (only two grace/late days permitted)
Fri, 6-Dec	Exam 3 Review OH

Important Notes

Tentative Schedule

Classification & Regression

Linear Models

Neural Networks

Societal Impacts

Learning Theory

Deep Learning

Reinforcement Learning

Learning Paradigms