10-418 + 10-618, Fall 2019

Important Notes

This schedule is tentative and subject to change. Please check back often.

Lecture Videos

You can view all the videos (both past and scheduled live streams) here: https://www.youtube.com/channel/UCF4lXfWfgTvslz6yNGVkQTg
We will include links to individual videos/live streams on the schedule below.
See Piazza note "Welcome to the Live Stream!" for additional details.

Tentative Schedule

Date	Lecture	Readings	Announcements
Search-based Structured Prediction
Mon, 26-Aug	Lecture 1 : Course Introduction [Slides] [Whiteboard] [Video]
Wed, 28-Aug	Lecture 2 : Reducing Multiclass to Binary Classification [Slides] [Whiteboard] [Video]	Beyond Binary Classification. Hal Daumé III (2017). CIML, Chapter 6.
Fri, 30-Aug	(No Recitation)
Mon, 2-Sep	(No Class: Labor Day)
Wed, 4-Sep	Lecture 3 : Structured Prediction as Search [Slides] [Whiteboard] [Video]	Imitation Learning. Hal Daumé III (2017). CIML, Chapter 18.
Fri, 6-Sep	Recitation: PyTorch
Mon, 9-Sep	Lecture 4 : Learning to Search / Recurrent neural networks (RNNs) [Slides] [Whiteboard] [Video]	Sequence Modeling: Recurrentand Recursive Nets. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.1-10.4.
Wed, 11-Sep	Lecture 5 : Sequence-to-sequence Models [Slides] [Whiteboard] [Video]	Sequence Modeling: Recurrent and Recursive Nets. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.	HW1 out (Thu)
Fri, 13-Sep	Recitation: HW1
Graphical Models: Representation
Mon, 16-Sep	Lecture 6 : Locally Normalized Models: Bayesian Networks [Slides] [Whiteboard] [Video]	Conditional Independence and Factorization. Michael I. Jordon (2003). An Introduction to Probabilistic Graphical Models, Chapter 2 (Section 2.1).
Wed, 18-Sep	Lecture 7 : Globally Normalized Models: Markov Random Fields & Conditional Random Fields [Slides] [Whiteboard] [Video]	An Introduction to Conditional Random Fields. Sutton and McCallum (2010). Conditional Independence and Factorization. Michael I. Jordon (2003). An Introduction to Probabilistic Graphical Models, Chapter 2 (Section 2.2 - end).
Fri, 20-Sep	(No Recitation)
Mon, 23-Sep	Lecture 8 : Factor Graphs / Exact Marginal Inference: Variable Elimination [Slides] [Whiteboard] [Video]	The Elimination Algorithm. Michael I. Jordon (2003). An Introduction to Probabilistic Graphical Models, Chapter 3. Probability Propagation and Factor Graphs. Michael I. Jordon (2003). An Introduction to Probabilistic Graphical Models, Chapter 4. Inference in Graphical Models. Nowozin and Lampert (2011). Structured Learning and Prediction in Computer Vision, Chapter 3 (Section 3.1).
Graphical Models: Exact Inference and Learning
Wed, 25-Sep	Lecture 9 : Exact Marginal/MAP Inference: Belief Propagation [Slides] [Whiteboard] [Video]	An Introduction to Conditional Random Fields. Sutton and McCallum (2010).	HW1 due (Thu)
Fri, 27-Sep	(No Recitation)		HW2 out (Sat)
Mon, 30-Sep	Lecture 10 : Learning fully observable MRFs and CRFs [Slides] [Whiteboard] [Video]	Bidirectional LSTM-CRF Models for Sequence Tagging. Huang, Xu, and Yu (2015).
Wed, 2-Oct	Lecture 11 : Neural Potential Functions [Slides] [Whiteboard] [Video]	Inference in Graphical Models. Nowozin and Lampert (2011). Structured Learning and Prediction in Computer Vision, Chapter 3 (Section 3.2.2).
Fri, 4-Oct	Recitation: HW2
Mon, 7-Oct	Lecture 12 : MAP Inference: Mixed Integer Linear Programming (Part I) [Slides] [Whiteboard] [Video]	[(Section 3 only)] A Tutorial Guide to Mixed-Integer Programming Models and Solution Techniques. Smith & Taskin (2008). Optimization in Medicine and Biology.
Wed, 9-Oct	Lecture 13 : MAP Inference: Mixed Integer Linear Programming (Part II) [Slides] [Whiteboard] [Video]	Integer Linear Programming Inference for Conditional Random Fields. Roth and Yih (2005). ICML.
Fri, 11-Oct	Recitation: Midterm Exam Review		HW2 due (Sat)
Learning for Structured Prediction
Mon, 14-Oct	Lecture 14 : Midterm Exam Review / Structured Perceptron [Slides] [Whiteboard] [Video]	Structured Prediction. Hal Daumé III (2017). CIML, Chapter 17 (Sections 17 - 17.3).
Wed, 16-Oct	Lecture 15 : Structured SVM [Slides] [Whiteboard] [Video]	Structured Support Vector Machines. Nowozin and Lampert (2011). Structured Learning and Prediction in Computer Vision, Chapter 6.
Thu, 17-Oct	Midterm Exam (evening exam -- details will be announced on Piazza)
Fri, 18-Oct	(No class: Mid-semester break)
Approximate Inference: MCMC
Mon, 21-Oct	Lecture 16 : Convolutional neural networks (CNNs) [Slides] [Whiteboard] [Video]	Convolutional Networks. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 9.1 - 9.3.
Wed, 23-Oct	Lecture 17 : Monte Carlo Methods [Slides] [Whiteboard] [Video]	Monte Carlo Methods. Li (2003). Information Theory, Inference, and Learning Algorithms, Chapter 29 (Section 29.1-29.3).	Project team due
Thu, 24-Oct	Recitation: HW3 (rescheduled to Mon, Oct-28 6:30pm-7:30pm in GHC 4401)		HW3 out
Fri, 25-Oct	(No class: Day for community engagement)
Mon, 28-Oct	Lecture 18 : Markov Chain Monte Carlo: Gibbs Sampling & Metropolis-Hastings (Recitation: HW3 at 6:30pm-7:30pm in GHC 4401) [Slides] [Whiteboard] [Video]	Monte Carlo Methods. Li (2003). Information Theory, Inference, and Learning Algorithms, Chapter 29 (Section 29.4 - 29.5).
Wed, 30-Oct	Lecture 19 : Markov Chains [Slides] [Whiteboard] [Video]	Monte Carlo Methods. Li (2003). Information Theory, Inference, and Learning Algorithms, Chapter 29 (Section 29.6 - 29.10).
Fri, 1-Nov	10-618 Project Team Office Hours
Approximate Inference: Variational Methods
Mon, 4-Nov	Lecture 20 : Bayesian Inference for Parameter Estimation [Slides] [Whiteboard] [Video]	Probability Propagation and Factor Graphs. Michael I. Jordon (2003). An Introduction to Probabilistic Graphical Models, Chapter 5.
Wed, 6-Nov	Lecture 21 : Topic Modeling [Slides] [Whiteboard] [Video]		HW4 out HW3 due
Fri, 8-Nov	Recitation: HW4		Project team due (Thu, Nov-07)
Mon, 11-Nov	Lecture 22 : Mean Field Variational Inference (Part I) [Slides] [Whiteboard] [Video]	Variational Inference: A Review for Statisticians. Blei, Kucukelbir, and McAuliffe (2018).	Project proposal due
Wed, 13-Nov	Lecture 23 : Mean Field Variational Inference (Part II) [Slides] [Whiteboard] [Video]
Fri, 15-Nov	10-618 Project Team Office Hours
Mon, 18-Nov	Lecture 24 : Coordinate Ascent Variational Inference [Slides] [Whiteboard] [Video]		HW4 due
Advanced Topics
Wed, 20-Nov	Lecture 25 : Learning partially observable graphical models / Variational EM [Slides] [Whiteboard] [Video]	High-Level Explanation of Variational Inference. Jason Eisner (2011).	HW5 out
Fri, 22-Nov	Project midway poster session (Session I) [1:00pm - 3:00pm in GHC 6115]		Project midway poster due (Thu, Nov-21)
Mon, 25-Nov	Lecture 26 : Bayesian Nonparametrics / Project midway poster session (Session II) [7:30pm - 9:30pm in GHC 6115] [Slides] [Whiteboard] [Video]	A Tutorial on Bayesian Nonparametric Models. Gershman & Blei (2011).
Wed, 27-Nov	(No class: Thanksgiving break)
Fri, 29-Nov	(No class: Thanksgiving break)
Mon, 2-Dec	Recitation: Final Exam Review		HW5 due
Wed, 4-Dec	Lecture 27 : Variational Autoencoders / Restricted Boltzman Machines [Slides] [Whiteboard] [Video]	Deep Generative Models. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 20. Tutorial on Variational Autoencoders. Carl Doersch (2016).
Thu, 5-Dec	Final Exam (evening exam) [6:30pm - 9:00pm in DH A302]
Fri, 6-Dec	(No Recitation)
Wed, 11-Dec	Project final poster session (time/location TBD)		Project final poster due (Tue, Dec-10)

Machine Learning for Structured Data

Important Notes

Lecture Videos

Tentative Schedule

Search-based Structured Prediction

Graphical Models: Representation

Graphical Models: Exact Inference and Learning

Learning for Structured Prediction

Approximate Inference: MCMC

Approximate Inference: Variational Methods

Advanced Topics