10-301 + 10-601, Fall 2022

Important Notes

This schedule is tentative and subject to change. Please check back often.

Tentative Schedule

Date	Lecture	Readings	Announcements
Classification & Regression
Mon, 29-Aug	Lecture 1 : Course Overview [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)]	Command Line and File I/O Tutorial. 10601 Course Staff (2020). 10601 Learning Objectives. Matt Gormley (2021). Math Resources. 10601 Course Staff (2021).	HW1 Out
Wed, 31-Aug	Lecture 2 : Machine Learning as Function Approximation [Slides] [Slides (Inked)]	10601 Notation Crib Sheet. Matt Gormley (2018).
Fri, 2-Sep	Recitation: HW1 [Handout] [Solutions] [Whiteboard (OneNote)]
Mon, 5-Sep	Labor Day
Wed, 7-Sep	Lecture 3 : Decision Trees [Slides] [Slides (Inked)] [Poll]	Visual Information Theory. Christopher Olah (2015). blog. Decision Trees. Hal Daumé III (2017). CIML, Chapter 1.	HW1 Due HW2 Out
Thu, 8-Sep	Lecture 3.5 : Decision Trees - Pseudocode [Slides] [Video]
Fri, 9-Sep	Recitation: HW2 [Handout] [Solutions] [Whiteboard (OneNote)]
Mon, 12-Sep	Lecture 4 : k-Nearest Neighbors [Slides] [Slides (Inked)] [Poll]	Geometry and Nearest Neighbors. Hal Daumé III (2017). CIML, Chapter 3.	HW1 Solution Session- Tuesday
Wed, 14-Sep	Lecture 5 : Model Selection and Experimental Design [Slides] [Slides (Inked)] [Poll]	Limits of Learning. Hal Daumé III (2017). CIML, Chapter 2.
Fri, 16-Sep	(No Recitation)
Linear Models
Mon, 19-Sep	Lecture 6 : Perceptron [Slides] [Slides (Inked)] [Poll]	The Perceptron. Hal Daumé III (2017). CIML, Chapter 4.	HW2 Due
Wed, 21-Sep	Lecture 7 : Linear Regression [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Linear Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 7.1-7.3.	HW3 Out
Fri, 23-Sep	Recitation: HW3 [Handout] [Solutions] [Whiteboard (OneNote)]
Sat, 24-Sep			HW2 Solution Session
Mon, 26-Sep	Lecture 8 : Exam 1 Review / Optimization for ML [Slides] [Slides (Inked)] [Poll]	An overview of gradient descent optimization algorithms. Sebastian Ruder (2017).
Wed, 28-Sep	Lecture 9 : Stochastic Gradient Descent / Logistic Regression [Slides] [Slides (Inked)] [Poll]	[Required] Estimating Probabilities: MLE and MAP. Tom Mitchell (2016). [Optional] Probability and Information Theory. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 3. Logistic Regression. Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 1.4.6, 8.1-8.3, 8.6.	HW3 due (only two grace/late days permitted) Exam 1 practice problems out
Fri, 30-Sep	Lecture 10 : Feature Engineering / Regularization [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Regularization for Deep Learning. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 7.1 and 7.8.
Sun, 2-Oct			HW3 Solution Session
Deep Learning
Mon, 3-Oct	Lecture 11 : Neural Networks [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Deep Feedforward Networks. Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016). Deep Learning, Chapter 6.1-6.4.
Tue, 4-Oct	Exam 1 (evening exam, details will be announced on Piazza)		HW4 Out
Wed, 5-Oct	Lecture 12 : Neural Networks + Backpropagation [Slides] [Slides (Inked)] [Whiteboard (OneNote)] [Poll]	Deep Feedforward Networks. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 6.5. Matrix Calculus for 10-301/601. Hoeseong (Hayden) Kim, Abhishek Vijayakumar (2022). 10601 Course Staff.
Fri, 7-Oct	Recitation: HW4 [Handout] [Solutions] [Whiteboard (OneNote)]
Mon, 10-Oct	Lecture 13 : Backpropagation [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	[Optional] Deep learning. Yann LeCun, Yoshua Bengio, & Geoffrey Hinton (2015). Nature.
Wed, 12-Oct	Lecture 14 : Deep Learning [Slides] [Slides (Inked)] [Whiteboard (OneNote)] [Poll]	Numpy Notes. 10601 Course Staff (2021). Logging Tutorial. 10601 Course Staff (2021). Workflow and Debugging Slides. 10601 Course Staff (2021).
Thu, 13-Oct			HW4 Due HW5 Out
Fri, 14-Oct	Recitation: HW5 [Handout] [Solutions] [Whiteboard (OneNote)]
Mon, 17-Oct	Fall break
Tue, 18-Oct
Wed, 19-Oct	Fall break
Thu, 20-Oct
Fri, 21-Oct	Fall break
Learning Theory
Mon, 24-Oct	Lecture 15 : PAC Learning [Slides] [Slides (Inked)] [Poll]	Generalization Abilities: Sample Complexity Results.. Nina Balcan (2015). Lecture notes.
Tue, 25-Oct			HW4 Solution Session
Wed, 26-Oct	Lecture 16 : PAC Learning / MLE+MAP [Slides] [Slides (Inked)] [Poll]	[Required] Estimating Probabilities: MLE and MAP. Tom Mitchell (2016). [Optional] Probability and Information Theory. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 3. A Few Useful Things to Know about Machine Learning. Pedro Domingos (2012).
Thu, 27-Oct			HW5 Due HW6 Out
Fri, 28-Oct	Tartan Community Day
Mon, 31-Oct	Lecture 17 : Naive Bayes [Slides] [Slides (Inked)] [Poll]	Generative and Discriminative Classifiers: Naive Bayes and Logistic Regression. Tom Mitchell (2016, draft).
Tue, 1-Nov	Lecture 17.5 : Naive Bayes Predictions+MAP [Slides] [Video]		HW5 Solution Session
Wed, 2-Nov	Recitation: HW6 [Handout] [Solutions] [Whiteboard (OneNote)]
Graphical Models
Fri, 4-Nov	Lecture 18 : Exam 2 Review / Hidden Markov Models (Part I) [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	[Only pages 257 - 266] A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Rabiner (1989).	HW6 Due (only two grace/late days permitted) Exam 2 practice problems out
Mon, 7-Nov	Lecture 19 : Hidden Markov Models (Part II) [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	[Only pages 257 - 266] A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Rabiner (1989).
Tue, 8-Nov			HW6 Solution Session
Wed, 9-Nov	Lecture 20 : Bayesian Networks [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Directed Graphical Models (Bayes nets). Kevin P. Murphy (2014). Machine Learning: A Probabilistic Perspective. Chapter 10.
Thu, 10-Nov	Exam 2 (evening exam, details will be announced on Piazza)
Fri, 11-Nov	Recitation: HW7 [Handout] [Solutions] [Whiteboard (OneNote)]		HW7 Out
Reinforcement Learning
Mon, 14-Nov	Lecture 21 : Reinforcement Learning: MDPs [Slides] [Slides (Inked)] [Poll]	Reinforcement Learning: A Survey. Kaelbling, et al (1996).
Wed, 16-Nov	Lecture 22 : Reinforcement Learning: Value/Policy Iteration [Slides] [Slides (Inked)] [Poll]	Reinforcement Learning: A Survey. Kaelbling, et al (1996).
Fri, 18-Nov	Recitation: HW8 [Handout] [Solutions] [Whiteboard (OneNote)]
Mon, 21-Nov	Lecture 23 : Reinforcement Learning: Q-Learning / Deep RL [Slides] [Slides (Inked)] [Poll]	[Optional] Playing Atari with Deep Reinforcement Learning. Mnih, et al (2013).	HW7 Due HW8 Out
Wed, 23-Nov	Thanksgiving Holiday- No class
Thu, 24-Nov	Thanksgiving Holiday- No class
Fri, 25-Nov	Thanksgiving Holiday- No class
Learning Paradigms
Mon, 28-Nov	Lecture 24 : Dimensionality Reduction: PCA [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	A Tutorial on Principal Component Analysis. Jonathon Shlens (2014).
Tue, 29-Nov			HW7 Solution Session
Wed, 30-Nov	Lecture 25 : Ensemble Methods / Recommender Systems [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	A Brief Introduction to Boosting. Robert E. Schapire (1999). Matrix Factorization Techniques for Recommender Systems. Koren, Bell, and Volinsky (2009).
Fri, 2-Dec	Recitation: HW9 [Handout] [Solutions] [Whiteboard (OneNote)]		HW8 Due HW9 Out
Mon, 5-Dec	Lecture 26 : K-Means / Societal Impacts of ML [Slides] [Slides (Inked)] [Whiteboard (OneNote)] [Poll]	k-means++: The Advantages of Careful Seeding. Arthur & Vassilvitskii (2006).	Exam 3 practice problems out
Wed, 7-Dec	Lecture 27 : Significance Testing for ML / Exam 3 Review / Course Overview [Slides] [Slides (Inked)] [Whiteboard (OneNote)] [Poll]	10601 Learning Objectives. Matt Gormley (2021).	HW8 Solution Session
Fri, 9-Dec	(No Recitation)		HW9 due (only two grace/late days permitted)
Sun, 11-Dec			HW9 Solution Session
Thu, 15-Dec	Exam 3 (9:30am-11:30am -- details will be announced on Piazza)

Introduction to Machine Learning

Important Notes

Tentative Schedule

Classification & Regression

Linear Models

Deep Learning

Learning Theory

Graphical Models

Reinforcement Learning

Learning Paradigms