Introduction to Machine Learning

Important Notes

This schedule is tentative and subject to change. Please check back often.

Tentative Schedule

Date	Lecture	Readings	Announcements
Classification & Regression
Wed, 17-Jan	Lecture 1 : Course Overview [Slides] [Slides (Inked)]	Command Line and File I/O Tutorial. 10601 Course Staff (2020). 10601 Learning Objectives. Matt Gormley (2023). Math Resources. 10601 Course Staff (2023).	HW1 Out
Fri, 19-Jan	Recitation: HW1 [Handout] [Solutions]
Mon, 22-Jan	Lecture 2 : Machine Learning as Function Approximation [Slides] [Slides (Inked)]	10601 Notation Crib Sheet. Matt Gormley (2023).
Wed, 24-Jan	Lecture 3 : Decision Trees [Slides] [Slides (Inked)] [Poll]	Visual Information Theory. Christopher Olah (2015). blog. Decision Trees. Hal Daumé III (2017). CIML, Chapter 1.	HW1 Due HW2 Out
Fri, 26-Jan	Recitation: HW2 [Handout] [Solutions]
Mon, 29-Jan	Lecture 4 : k-Nearest Neighbors [Slides] [Slides (Inked)] [Poll]	Geometry and Nearest Neighbors. Hal Daumé III (2017). CIML, Chapter 3.
Wed, 31-Jan	Lecture 5 : Model Selection and Experimental Design [Slides] [Slides (Inked)] [Poll]	Limits of Learning. Hal Daumé III (2017). CIML, Chapter 2.
Fri, 2-Feb	(No Recitation)
Linear Models
Mon, 5-Feb	Lecture 6 : Perceptron [Slides] [Slides (Inked)] [Poll]	The Perceptron. Hal Daumé III (2017). CIML, Chapter 4.	HW2 Due HW3 Out
Wed, 7-Feb	Lecture 7 : Linear Regression [Slides] [Slides (Inked)] [Poll]	Linear Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 7.1-7.3.
Fri, 9-Feb	Recitation: HW3 [Handout] [Solutions]
Mon, 12-Feb	Lecture 8 : Optimization for ML [Slides] [Slides (Inked)] [Poll]	An overview of gradient descent optimization algorithms. Sebastian Ruder (2017).	HW3 Due (only two grace/late days permitted) Exam 1 Practice Problems out
Wed, 14-Feb	Lecture 9 : Stochastic Gradient Descent / Logistic Regression [Slides] [Slides (Inked)] [Poll]	Estimating Probabilities: MLE and MAP. Tom Mitchell (2016). Probability and Information Theory. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 3. Logistic Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 1.4.6, 8.1-8.3, 8.6.
Fri, 16-Feb	Exam 1 Review OH
Mon, 19-Feb	Lecture 10 : Feature Engineering / Regularization [Slides] [Slides (Inked)] [Poll]	Regularization for Deep Learning. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 7.1 and 7.8.
Mon, 19-Feb	Exam 1 (evening exam, details will be announced on Piazza) 7p - 9p		HW4 Out
Neural Networks
Wed, 21-Feb	Lecture 11 : Neural Networks [Slides] [Slides (Inked)] [Poll]	Deep Feedforward Networks. Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016). Deep Learning, Chapter 6.1-6.4.
Fri, 23-Feb	Recitation: HW4 [Handout] [Solutions]
Mon, 26-Feb	Lecture 12 : Backpropagation I [Slides] [Slides (Inked)] [Poll]	Deep Feedforward Networks. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 6.5. Matrix Calculus for 10-301/601. Hoeseong (Hayden) Kim, Abhishek Vijayakumar (2022). 10601 Course Staff.
Wed, 28-Feb	Lecture 13 : Backpropagation II [Slides] [Slides (Inked)] [Poll]	Numpy Notes. 10601 Course Staff (2021). Logging Tutorial. 10601 Course Staff (2021). Workflow and Debugging Slides. 10601 Course Staff (2021).	HW4 Due HW5 Out
Fri, 1-Mar	Recitation: HW5 [Handout] [Solutions]
Mon, 4-Mar	(Spring Break - No Class)
Tue, 5-Mar
Wed, 6-Mar	(Spring Break - No Class)
Thu, 7-Mar
Fri, 8-Mar	(Spring Break - No Class)
Learning Theory
Mon, 11-Mar	Lecture 14 : PAC Learning [Slides] [Slides (Inked)] [Poll]	Generalization Abilities: Sample Complexity Results.. Nina Balcan (2015). Lecture notes.
Societal Impacts
Wed, 13-Mar	Lecture 15 : PAC Learning / Societal Impacts of ML I [Slides] [Slides (Inked)] [Poll]	A Few Useful Things to Know about Machine Learning. Pedro Domingos (2012).
Fri, 15-Mar	Recitation: HW6 [Handout] [Solutions]
Mon, 18-Mar	Lecture 16 : Societal Impacts of ML II [Slides] [Poll]		HW5 Due HW6 Out, Exam 2 Practice Problems out
Deep Learning
Wed, 20-Mar	Lecture 17 : Foundations: CNNs [Slides] [Slides (Inked)] [Poll]	Deep learning. Yann LeCun, Yoshua Bengio, & Geoffrey Hinton (2015). Nature.
Fri, 22-Mar	Exam 2 Review OH
Sun, 24-Mar			HW6 Due (only two grace/late days permitted)
Mon, 25-Mar	Lecture 18 : Foundations: RNNs [Slides] [Slides (Inked)] [Poll]
Wed, 27-Mar	Lecture 19 : Transformers and Autodiff [Slides] [Slides (Inked)] [Poll]
Thu, 28-Mar	Exam 2 (evening exam, details will be announced on Piazza)		HW7 Out
Fri, 29-Mar	Recitation: HW7 [Handout] [Solutions] [Supplemental Material]
Reinforcement Learning
Mon, 1-Apr	Lecture 20 : Reinforcement Learning: MDPs [Slides] [Slides (Inked)] [Poll]	Reinforcement Learning: A Survey. Kaelbling, et al (1996).
Wed, 3-Apr	Lecture 21 : Reinforcement Learning: Value/Policy Iteration [Slides] [Slides (Inked)] [Poll]	Reinforcement Learning: A Survey. Kaelbling, et al (1996).
Fri, 5-Apr	Lecture 22 : Reinforcement Learning: Q-Learning / Deep RL [Slides] [Slides (Inked)] [Poll]	Playing Atari with Deep Reinforcement Learning. Mnih, et al (2013).
Mon, 8-Apr	Recitation: HW8 [Handout] [Solutions]		HW7 Due HW8 Out
Learning Paradigms
Wed, 10-Apr	Lecture 23 : Dimensionality Reduction: PCA [Slides] [Slides (Inked)] [Poll]	A Tutorial on Principal Component Analysis. Jonathon Shlens (2014).
Thu, 11-Apr	(Spring Carnival)
Fri, 12-Apr	(Spring Carnival - No Class)
Sat, 13-Apr	(Spring Carnival)
Mon, 15-Apr	Lecture 24 : K-Means / Ensemble Methods: Bagging [Slides] [Slides (Inked)] [Poll]	k-means++: The Advantages of Careful Seeding. Arthur & Vassilvitskii (2006). Ensemble Methods. Hal Daumé III (2017). CIML, Chapter 13.
Wed, 17-Apr	Lecture 25 : Ensemble Methods: Boosting / Recommender Systems [Slides] [Slides (Inked)] [Poll]	A Brief Introduction to Boosting. Robert E. Schapire (1999). Matrix Factorization Techniques for Recommender Systems. Koren, Bell, and Volinsky (2009).
Fri, 19-Apr	Recitation: HW9 [Handout] [Solutions]		HW8 Due HW9 Out
Mon, 22-Apr	Lecture 26 : Special Topics: Pretraining, Fine-tuning, In-context Learning [Slides] [Poll]		Exam 3 Practice Problems out
Wed, 24-Apr	Lecture 27 : Special Topics: Generative Models for Vision / Significance Testing for ML [Slides] [Poll]	10601 Learning Objectives. Matt Gormley (2023).
Thu, 25-Apr			HW9 Due (only two grace/late days permitted)
Fri, 26-Apr	Exam 3 Review OH
Tue, 30-Apr	Exam 3 (details will be announced on Piazza) 9:30a - 11:30a

Important Notes

Tentative Schedule

Classification & Regression

Linear Models

Neural Networks

Learning Theory

Societal Impacts

Deep Learning

Reinforcement Learning

Learning Paradigms