10-301 + 10-601, Spring 2021

Important Notes

This schedule is tentative and subject to change. Please check back often.

Tentative Schedule

Date	Lecture	Readings	Announcements
Classification & Regression
Mon, 1-Feb	Lecture 1 : Course Overview [Slides]	10601 Notation Crib Sheet. Matt Gormley (2018). Command Line and File I/O Tutorial. 10601 Course Staff (2020). 10601 Learning Objectives. Matt Gormley (2018). Visual Information Theory. Christopher Olah (2015). blog.
Wed, 3-Feb	Lecture 2 : Decision Trees, Overfitting [Slides]	Decision Trees. Hal Daumé III (2017). CIML, Chapter 1.	HW1 out
Fri, 5-Feb	Recitation: HW1 [Handout] [Solutions]
Mon, 8-Feb	Lecture 3 : Generalizing from exampes - the Big Picture [Slides] [Poll]	Limits of Learning. Hal Daumé III (2017). CIML, Chapter 2.
Wed, 10-Feb	Lecture 4 : k-Nearest Neighbors [Slides] [Whiteboard] [Poll]	Geometry and Nearest Neighbors. Hal Daumé III (2017). CIML, Chapter 3.	HW1 due HW2 out
Fri, 12-Feb	Recitation: HW2 [Handout] [Solutions]
Mon, 15-Feb	Lecture 5 : Model Selection [Slides] [Whiteboard] [Poll]
Wed, 17-Feb	Lecture 6 : Perceptron [Slides] [Whiteboard] [Poll]	The Perceptron. Hal Daumé III (2017). CIML, Chapter 4.	HW1 solution session (Thursday)
Fri, 19-Feb	Recitation: Colab / Linear Algebra Libraries / Debugging (Team D) [Slides] [Handout]
Linear Models
Mon, 22-Feb	Lecture 7 : Linear Regression [Slides] [Whiteboard] [Poll]	Linear Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 7.1-7.3.	HW2 due HW3 out
Wed, 24-Feb	Lecture 8 : Optimization for ML [Slides] [Whiteboard] [Poll]	An overview of gradient descent optimization algorithms. Sebastian Ruder (2017).
Fri, 26-Feb	Recitation: HW3 [Handout] [Solutions]
Mon, 1-Mar	Lecture 9 : Midterm Exam Review / Binary Logistic Regression [Slides] [Whiteboard] [Poll]	[Required] Estimating Probabilities: MLE and MAP. Tom Mitchell (2016). [Optional] Probability and Information Theory. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 3. Logistic Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 1.4.6, 8.1-8.3, 8.6.	HW3 due Exam 1 practice problems out HW2 Solution Session
Wed, 3-Mar	Lecture 10 : Feature Engineering / Regularization [Slides] [Whiteboard] [Poll]	Regularization for Deep Learning. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 7.1 and 7.8.	Mock Exam 1
Fri, 5-Mar	No recitation		HW3 Solutions Session
Sat, 6-Mar	Midterm Exam 1 (Sat)-- details will be announced on Piazza
Deep Learning
Mon, 8-Mar	Lecture 11 : Deep learning, Neural networks, Backpropagation [Slides] [Poll]	Deep Feedforward Networks. Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016). Deep Learning, Chapter 6.1-6.4.	HW4 out (Mon)
Wed, 10-Mar	Lecture 12 : Representation learning, word embeddings, CNNs [Slides] [Poll]	Deep Feedforward Networks. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 6.5.
Fri, 12-Mar	Recitation: HW4 [Handout] [Solutions]
Mon, 15-Mar	Lecture 13 : Recurrent networks, LSTMs, Seq2Seq models [Slides] [Supplemental Material] [Poll]
Wed, 17-Mar	Lecture 14 : Advanced Architectures, Attention, Transformers, GPT [Slides] [Poll]	[Optional] Deep learning. Yann LeCun, Yoshua Bengio, & Geoffrey Hinton (2015). Nature.	HW4 due (Thurs) HW5 out (Thurs)
Fri, 19-Mar	No classes: Mid-Semester Break
Mon, 22-Mar	Recitation: HW5 [Handout] [Solutions]
Reinforcement Learning
Wed, 24-Mar	Lecture 15 : Reinforcement Learning: MDPs + Value/Policy Iteration [Slides] [Whiteboard] [Poll]	Reinforcement Learning: A Survey. Kaelbling, et al (1996).	HW4 Solutions Session (Thurs)
Fri, 26-Mar	Lecture 16 : Reinforcement Learning: Q-Learning [Slides] [Poll]
Mon, 29-Mar	Lecture 17 : Deep Reinforcement Learning [Slides] [Poll]	[Optional] Playing Atari with Deep Reinforcement Learning. Mnih, et al (2013).	HW5 due HW6 out
Generative Models
Wed, 31-Mar	Lecture 18 : MLE/MAP [Slides] [Poll]	[Required] Estimating Probabilities: MLE and MAP. Tom Mitchell (2016). [Optional] Probability and Information Theory. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 3.
Fri, 2-Apr	Recitation: HW6 [Handout] [Solutions] [Supplemental Material] [Supplemental Solutions]		HW5 Solution Session
Mon, 5-Apr	No classes: Break Day
Wed, 7-Apr	Lecture 19 : Naive Bayes [Slides] [Poll]	Generative and Discriminative Classifiers: Naive Bayes and Logistic Regression. Tom Mitchell (2016, draft).	HW6 due Exam 2 practice problems out
Fri, 9-Apr	Mock Exam 2
Probabilistic Graphical Models
Mon, 12-Apr	Lecture 20 : PGM: Representation [Slides] [Poll]	Directed Graphical Models (Bayes nets). Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 10.	HW6 Solution Session
Tues, 13-Apr	Midterm Exam 2 (evening exam) -- details will be announced on Piazza
Wed, 14-Apr	Lecture 21 : PGM: Inference [Slides] [Poll]
Fri, 16-Apr	No classes: Carnival (April 15-17)
Mon, 19-Apr	Lecture 22 : PGM: Learning [Slides] [Poll]		HW7 out
Wed, 21-Apr	Lecture 23 : PGM: EM and Clustering [Slides] [Whiteboard] [Poll]
Fri, 23-Apr	Recitation: HW7 [Handout] [Solutions]
Learning Theory
Mon, 26-Apr	Lecture 24 : Learning Theory: PAC Learning [Slides] [Whiteboard] [Poll]	Generalization Abilities: Sample Complexity Results.. Nina Balcan (2015). Lecture notes.
Wed, 28-Apr	Lecture 25 : Learning Theory: PAC Learning [Slides] [Whiteboard] [Poll]	A Few Useful Things to Know about Machine Learning. Pedro Domingos (2012).
Learning Paradigms
Fri, 30-Apr	Lecture 26 : Ensemble Methods / Recommender Systems [Slides] [Poll]	A Brief Introduction to Boosting. Robert E. Schapire (1999). Matrix Factorization Techniques for Recommender Systems. Koren, Bell, and Volinsky (2009).	HW7 due HW8 out
Mon, 3-May	Lecture 27 : Dimensionality Reduction: PCA, Autoencoders, Matrix Factorization [Slides] [Whiteboard] [Poll]	A Tutorial on Principal Component Analysis. Jonathon Shlens (2014).
Wed, 5-May	Recitation: HW8 [Handout] [Solutions]
Fri, 7-May	Mock Exam 3		HW8 due Exam 3 practice problems out HW7 Solution Session
Fri, 14-May	Final Exam (Friday, May 14th at 9:30am) -- details will be announced on Piazza	10601 Learning Objectives. Matt Gormley (2018).	HW 8 Solution Session (Thurs)

Introduction to Machine Learning

Important Notes

Tentative Schedule

Classification & Regression

Linear Models

Deep Learning

Reinforcement Learning

Generative Models

Probabilistic Graphical Models

Learning Theory

Learning Paradigms