10-418 + 10-618, Fall 2022

Important Notes

This schedule is tentative and subject to change. Please check back often.

Tentative Schedule

Date	Lecture	Readings	Announcements
Search-based Structured Prediction
Mon, 29-Aug	Lecture 1 : Course Overview / What is Structured Prediction? [Slides] [Slides (Inked)] [Whiteboard (OneNote)]
Wed, 31-Aug	Lecture 2 : Recurrent neural networks (RNNs) / Module-based Automatic Differentiation [Slides] [Slides (Inked)] [Whiteboard (OneNote)]	Sequence Modeling: Recurrent and Recursive Nets. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.1-10.4. A Framework for the Cooperation of Learning Algorithms. Léon Bottou, Patrick Gallinari (1991). Advances in Neural Information Processing Systems. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Paszke et al. (2019). Advances in Neural Information Processing Systems.
Fri, 2-Sep	(No Recitation)
Mon, 5-Sep	(No Class: Labor Day)
Wed, 7-Sep	Lecture 3 : 1D CNNs / Sequence-to-sequence Models [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Natural Language Processing (Almost) from Scratch. Collobert et al. (2011). Journal of Machine Learning Research. Sequence Modeling: Recurrent and Recursive Nets. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.	HW1 out
Fri, 9-Sep	Recitation: HW1 [Handout] [Whiteboard (OneNote)]
Mon, 12-Sep	Lecture 4 : Learning to Search (Part I): MLE & Decoding for seq2seq / Imitation Learning [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Imitation Learning. Hal Daumé III (2017). CIML, Chapter 18.
Wed, 14-Sep	Lecture 5 : Learning to Search (Part II): Imitation Learning / Structured Prediction as Search [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Imitation Learning. Hal Daumé III (2017). CIML, Chapter 18.
Fri, 16-Sep	(No Recitation)		HW1 due
Sat, 17-Sep	Lecture 5.5 : Learning to Search (Part III): Imitation Learning for Structured Prediction [video recorded] [Slides] [Slides (Inked)] [Whiteboard (PDF)]		HW2 out
Graphical Models: Representation, Exact Inference, and Learning
Mon, 19-Sep	Lecture 6 : Directed Graphical Models / Undirected Graphical Models [Slides] [Whiteboard (OneNote)] [Poll]	Conditional Independence and Factorization. Michael I. Jordan (2003). An Introduction to Probabilistic Graphical Models, Chapter 2 (Section 2.1). An Introduction to Conditional Random Fields. Sutton and McCallum (2010). Conditional Independence and Factorization. Michael I. Jordan (2003). An Introduction to Probabilistic Graphical Models, Chapter 2 (Section 2.2 - end).
Wed, 21-Sep	Recitation: HW2 [Handout] [Whiteboard (OneNote)]		HW1 Solution Session
Fri, 23-Sep	Lecture 7 : DGM & UGM Conditional Independencies / Factor Graphs [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Conditional Independence and Factorization. Michael I. Jordan (2003). An Introduction to Probabilistic Graphical Models, Chapter 2 (Section 2.1). An Introduction to Conditional Random Fields. Sutton and McCallum (2010). Conditional Independence and Factorization. Michael I. Jordan (2003). An Introduction to Probabilistic Graphical Models, Chapter 2 (Section 2.2 - end).
Mon, 26-Sep	Lecture 8 : Exact Marginal/MAP Inference: Variable Elimination & Belief Propagation [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	The Elimination Algorithm. Michael I. Jordan (2003). An Introduction to Probabilistic Graphical Models, Chapter 3.
Wed, 28-Sep	Lecture 9 : Belief Propagation / Learning fully observable MRFs [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Probability Propagation and Factor Graphs. Michael I. Jordan (2003). An Introduction to Probabilistic Graphical Models, Chapter 4. Inference in Graphical Models. Nowozin and Lampert (2011). Structured Learning and Prediction in Computer Vision, Chapter 3 (Section 3.1). Inference in Graphical Models. Nowozin and Lampert (2011). Structured Learning and Prediction in Computer Vision, Chapter 3 (Section 3.2.2).
Thu, 29-Sep			HW2 due HW3 out
Fri, 30-Sep	(No Recitation) [Whiteboard (OneNote)]
Mon, 3-Oct	Lecture 10 : Learning fully observable CRFs / Neural Potential Functions / MBR Decoding [Slides] [Slides (Inked)] [Whiteboard (OneNote)] [Poll]	An Introduction to Conditional Random Fields. Sutton and McCallum (2010). Bidirectional LSTM-CRF Models for Sequence Tagging. Huang, Xu, and Yu (2015).	HW2 Solution Session
Tue, 4-Oct	Recitation: HW3 (evening) [Handout] [Solutions]
Approximate Inference: MCMC
Wed, 5-Oct	Lecture 11 : Complexity of Inference / Monte Carlo Methods [Slides] [Slides (Inked)] [Whiteboard (OneNote)] [Poll]	Monte Carlo Methods. Li (2003). Information Theory, Inference, and Learning Algorithms, Chapter 29 (Section 29.1-29.3).
Fri, 7-Oct	(No Recitation)
Mon, 10-Oct	Lecture 12 : Midterm Exam Review / Markov Chain Monte Carlo: Gibbs Sampling & Metropolis-Hastings [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Monte Carlo Methods. Li (2003). Information Theory, Inference, and Learning Algorithms, Chapter 29 (Section 29.4 - 29.5).	HW3 due (only two grace/late days permitted) Practice Exam out
Wed, 12-Oct	Lecture 13 : Markov Chains / Bayesian Inference for Parameter Estimation [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]	Monte Carlo Methods. Li (2003). Information Theory, Inference, and Learning Algorithms, Chapter 29 (Section 29.6 - 29.10).
Fri, 14-Oct	Midterm Exam
Mon, 17-Oct	Fall break
Tue, 18-Oct
Wed, 19-Oct	Fall break
Thu, 20-Oct
Fri, 21-Oct	Fall break
Mon, 24-Oct	Lecture 14 : Bayesian Inference for Parameter Estimation / Topic Modeling [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]	Probability Propagation and Factor Graphs. Michael I. Jordan (2003). An Introduction to Probabilistic Graphical Models, Chapter 5.	HW4 out
Mon, 24-Oct	Recitation: HW4 (evening recitation, 6pm, GHC 6121) [Handout] [Solutions]
Wed, 26-Oct	Lecture 15 : Topic Modeling / Convolutional Neural Networks [Slides] [Slides (Inked)] [Poll]	Convolutional Networks. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 9.1 - 9.3. Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey. Jelodar et al. (2019). Multimedia Tools and Applications.
Fri, 28-Oct	Tartan Community Day
Approximate Inference: Variational Methods
Mon, 31-Oct	Lecture 16 : Mean Field Variational Inference [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]	Variational Inference: A Review for Statisticians. Blei, Kucukelbir, and McAuliffe (2018).
Wed, 2-Nov	Lecture 17 : Coordinate Ascent Variational Inference [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]	Variational Inference: A Review for Statisticians. Blei, Kucukelbir, and McAuliffe (2018).
Fri, 4-Nov	Lecture 18 : CAVI / Expectation Maximization [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]	High-Level Explanation of Variational Inference. Jason Eisner (2011).	HW4 due HW5 out
Mon, 7-Nov	Recitation: HW5 [Handout]
Wed, 9-Nov	Lecture 19 : Variational EM / Hidden State CRFs [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]	High-Level Explanation of Variational Inference. Jason Eisner (2011). Tutorial on Variational Autoencoders. Carl Doersch (2016).
Fri, 11-Nov	(No recitation)
Mon, 14-Nov	Lecture 20 : Variational Autoencoders [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]	Deep Generative Models. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 20.
Advanced Topics
Wed, 16-Nov	Lecture 21 : MAP Inference: Mixed Integer Linear Programming [Slides] [Whiteboard (PDF)] [Poll]	[(Section 3 only)] A Tutorial Guide to Mixed-Integer Programming Models and Solution Techniques. Smith & Taskin (2008). Optimization in Medicine and Biology. Integer Linear Programming Inference for Conditional Random Fields. Roth and Yih (2005). ICML.	HW5 due HW6 out
Fri, 18-Nov	Recitation: HW6 [Handout]
Mon, 21-Nov	Lecture 22 : Structured Perceptron / Structured SVM [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]	Convolutional Networks. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 9.1 - 9.3. Structured Prediction. Hal Daumé III (2017). CIML, Chapter 17 (Sections 17 - 17.3). Structured Support Vector Machines. Nowozin and Lampert (2011). Structured Learning and Prediction in Computer Vision, Chapter 6.
Wed, 23-Nov	Thanksgiving Holiday- No class
Thu, 24-Nov	Thanksgiving Holiday- No class
Fri, 25-Nov	Thanksgiving Holiday- No class
Mon, 28-Nov	Lecture 23 : Causal Inference [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]	Causal Inference in Statistics: A Primer. Pearl, Glymour, & Jewell (2016). Causation, Prediction, and Search. Spirtes et al. (1993). Learning causality and causality-related learning: some recent progress. Zhang et al. (2017).	Mini-Project out
Wed, 30-Nov	Lecture 24 : Causal Inference / Bayesian Nonparametrics [Slides] [Slides (Inked)] [Poll]	A Tutorial on Bayesian Nonparametric Models. Gershman & Blei (2011). Regression. Rasmussen & Williams (2006). Gaussian Processes for Machine Learning, Chapter 2. A Visual Exploration of Gaussian Processes. Görtler, Kehlbeck, & Deussen (2019).	HW6 due
Fri, 2-Dec	(No recitation)
Mon, 5-Dec	(Lecture rescheduled to Friday)		Practice Exam out
Wed, 7-Dec	Lecture 25 : Bayesian Nonparametrics / Graph Neural Networks [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]	A Gentle Introduction to Graph Neural Networks. Sanchez-Lengeling et al. (2021). A Comprehensive Survey on Graph Neural Networks. Wu et al. (2019).
Fri, 9-Dec	Lecture 26 : Graph Neural Networks / Final Exam Review [Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]		Mini-Project due
Thu, 15-Dec	Final Exam (5:30 pm - 7:30 pm, DH A302)

Machine Learning for Structured Data

Important Notes

Tentative Schedule

Search-based Structured Prediction

Graphical Models: Representation, Exact Inference, and Learning

Approximate Inference: MCMC

Approximate Inference: Variational Methods

Advanced Topics