10-301 + 10-601, Fall 2021

Important Notes

This schedule is tentative and subject to change. Please check back often.

Tentative Schedule

Date	Lecture	Readings	Announcements
Classification & Regression
Mon, 30-Aug	Lecture 1 : Course Overview [Slides] [Whiteboard]	10601 Notation Crib Sheet. Matt Gormley (2018). Command Line and File I/O Tutorial. 10601 Course Staff (2020). 10601 Learning Objectives. Matt Gormley (2021). Math Resources. 10601 Course Staff (2021).
Wed, 1-Sep	Lecture 2 : Decision Trees (Part I) [Slides] [Whiteboard]	Decision Trees. Hal Daumé III (2017). CIML, Chapter 1.	HW1 out
Fri, 3-Sep	Recitation: HW1 [Handout] [Solutions]
Mon, 6-Sep	Labor Day - No Classes
Wed, 8-Sep	Lecture 3 : Decision Trees (Part II) [Slides] [Whiteboard] [Poll]	Visual Information Theory. Christopher Olah (2015). blog. Limits of Learning. Hal Daumé III (2017). CIML, Chapter 2.	HW1 due HW2 out
Fri, 10-Sep	Recitation: HW2 [Handout] [Solutions]
Mon, 13-Sep	Lecture 4 : k-Nearest Neighbors [Slides] [Whiteboard] [Poll]	Geometry and Nearest Neighbors. Hal Daumé III (2017). CIML, Chapter 3.	HW1 Solution Session
Wed, 15-Sep	Lecture 5 : Model Selection [Slides] [Whiteboard] [Poll]
Fri, 17-Sep	Lecture 6 : Perceptron [Slides] [Whiteboard] [Poll]	The Perceptron. Hal Daumé III (2017). CIML, Chapter 4.
Linear Models
Mon, 20-Sep	Lecture 7 : Linear Regression [Slides] [Whiteboard] [Poll]	Linear Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 7.1-7.3.	HW2 due HW3 out
Wed, 22-Sep	Lecture 8 : Optimization for ML [Slides] [Whiteboard] [Poll]	An overview of gradient descent optimization algorithms. Sebastian Ruder (2017).
Fri, 24-Sep	Recitation: HW3 [Handout] [Solutions]
Sun, 26-Sep			HW3 due (only two grace days permitted) Mock Exam 1 and Exam 1 practice problems out
Mon, 27-Sep	Lecture 9 : Exam 1 Review / Stochastic Gradient Descent / Probabilistic Learning [Slides] [Whiteboard] [Poll]	[Required] Estimating Probabilities: MLE and MAP. Tom Mitchell (2016). [Optional] Probability and Information Theory. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 3. Logistic Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 1.4.6, 8.1-8.3, 8.6.	HW2 Solution Session
Wed, 29-Sep	Recitation: Colab / Linear Algebra Libraries / Debugging	Numpy Notes. 10601 Course Staff (2021). Logging Tutorial. 10601 Course Staff (2021). Workflow and Debugging Slides. 10601 Course Staff (2021).	HW3 Solution Session
Thu, 30-Sep	Exam 1 (evening exam, details will be announced on Piazza)
Fri, 1-Oct	Lecture 10 : Logistic Regression / Feature Engineering / Regularization [Slides] [Whiteboard] [Poll]	Regularization for Deep Learning. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 7.1 and 7.8.	HW4 out
Mon, 4-Oct	Recitation: HW4 [Handout] [Solutions]
Deep Learning
Wed, 6-Oct	Lecture 11 : Neural Networks [Slides] [Whiteboard] [Poll]	Deep Feedforward Networks. Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016). Deep Learning, Chapter 6.1-6.4.
Fri, 8-Oct	Lecture 12 : Backpropagation (Lecture switched with recitation) [Slides] [Whiteboard] [Poll]	Deep Feedforward Networks. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 6.5.
Mon, 11-Oct	Lecture 13 : Deep Learning [Slides] [Whiteboard] [Poll]	[Optional] Deep learning. Yann LeCun, Yoshua Bengio, & Geoffrey Hinton (2015). Nature.	HW4 due HW5 out
Learning Theory
Wed, 13-Oct	Lecture 14 : Learning Theory: PAC Learning [Slides] [Whiteboard] [Poll]	Generalization Abilities: Sample Complexity Results.. Nina Balcan (2015). Lecture notes.
Fri, 15-Oct	Recitation: HW5 [Handout] [Solutions]
Mon, 18-Oct	Lecture 15 : Learning Theory: PAC Learning [Slides] [Whiteboard] [Poll]	A Few Useful Things to Know about Machine Learning. Pedro Domingos (2012).	HW4 Solution Session
Generative Models
Wed, 20-Oct	Lecture 16 : MLE/MAP [Slides] [Whiteboard] [Poll]	[Required] Estimating Probabilities: MLE and MAP. Tom Mitchell (2016). [Optional] Probability and Information Theory. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 3.
Thu, 21-Oct			HW5 due HW6 out
Fri, 22-Oct	Recitation: HW6 [Handout] [Solutions]		Exam 2 practice problems out
Mon, 25-Oct	Lecture 17 : Naive Bayes [Slides] [Whiteboard] [Poll]	Generative and Discriminative Classifiers: Naive Bayes and Logistic Regression. Tom Mitchell (2016, draft).
Graphical Models
Wed, 27-Oct	Lecture 18 : Exam 2 Review / Hidden Markov Models (Part I) [Slides] [Whiteboard] [Poll]	[Only pages 257 - 266] A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Rabiner (1989).	HW5 Solution Session
Thu, 28-Oct			HW6 due (only two grace days permitted) Mock Exam 2 out (due Oct 31)
Fri, 29-Oct	Lecture 19 : Hidden Markov Models (Part II) (Lecture switched with recitation) [Slides] [Whiteboard] [Poll]
Mon, 1-Nov	Lecture 20 : Bayesian Networks [Slides] [Whiteboard] [Poll]	Directed Graphical Models (Bayes nets). Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 10.	HW6 Solution Session
Tue, 2-Nov	Exam 2 (evening exam, details will be announced on Piazza)
Wed, 3-Nov	Recitation: HW7 [Handout] [Solutions]		HW7 out
Fri, 5-Nov	No Classes - Day for Community Engagement
Reinforcement Learning
Mon, 8-Nov	Lecture 21 : Reinforcement Learning: MDPs [Slides] [Whiteboard] [Poll]	Reinforcement Learning: A Survey. Kaelbling, et al (1996).
Wed, 10-Nov	Lecture 22 : Reinforcement Learning: Value/Policy Iteration [Slides] [Whiteboard] [Poll]
Fri, 12-Nov	Lecture 23 : Reinforcement Learning: Q-Learning + Deep RL (Lecture switched with recitation) [Slides] [Whiteboard] [Poll]	[Optional] Playing Atari with Deep Reinforcement Learning. Mnih, et al (2013).	HW7 due HW8 out
Mon, 15-Nov	Recitation: HW8 [Handout] [Solutions]
Learning Paradigms
Wed, 17-Nov	Lecture 24 : K-Means / Dimensionality Reduction: PCA [Slides] [Whiteboard] [Poll]	A Tutorial on Principal Component Analysis. Jonathon Shlens (2014).
Thu, 18-Nov			HW7 Solution Session
Fri, 19-Nov	(No recitation)
Sun, 21-Nov			HW8 due HW9 out
Mon, 22-Nov	Lecture 25 : Ensemble Methods / Recommender Systems [Slides] [Whiteboard] [Poll]	A Brief Introduction to Boosting. Robert E. Schapire (1999). Matrix Factorization Techniques for Recommender Systems. Koren, Bell, and Volinsky (2009).
Tue, 23-Nov
Wed, 24-Nov	Thanksgiving Break - No Classes
Fri, 26-Nov	Thanksgiving Break - No Classes
Mon, 29-Nov	Recitation: HW9 [Handout] [Solutions]		Exam 3 practice problems out
Tue, 30-Nov			HW8 Solution Session
Wed, 1-Dec	Lecture 26 : Exam 3 Review / Societal Impacts of ML [Slides] [Whiteboard] [Poll]		HW9 due (only two grace days permitted) Mock Exam 3 out (due Dec 04)
Fri, 3-Dec	(No recitation)		HW9 Solution Session (Sun)
Mon, 6-Dec	Exam 3 (Monday, December 6th, 9:30am-11:30am) -- details will be announced on Piazza	10601 Learning Objectives. Matt Gormley (2021).

Introduction to Machine Learning

Important Notes

Tentative Schedule

Classification & Regression

Linear Models

Deep Learning

Learning Theory

Generative Models

Graphical Models

Reinforcement Learning

Learning Paradigms