10708 Probabilistic Graphical Models

Lecture Schedule

Lectures are held on Mondays and Wednesdays from 4:30-5:50 pm in GHC 4307.

All of the lecture videos can be found here.

Date	Lecture	Scribes	Readings	Videos
Monday, Jan 13	Lecture 1 (Eric) - Slides Introduction to Graphical Models	None	Required (no reading summary due): Koller and Friedman Textbook, Ch. 1 Koller and Friedman Textbook, Ch. 3 M. Jordan, An Introduction to Graphical Models, pg. 1-12	Video
Module 1: Representation
Wednesday, Jan 15	Lecture 2 (Eric) - Slides Directed Graphical Models: Bayesian Networks	Calvin McCarter, Daniel Ribeiro Silva (Scribe Notes)	Required: Koller and Friedman Textbook, Ch. 4 M. Jordan, An Introduction to Graphical Models, pg. 13-18	Video
Wednesday, Jan 22	Lecture 3 (Eric) - Slides Undirected Graphical Models: Markov Random Fields	Kirstin Early, Nicole Rafidi (Scribe Notes)	Required: None Optional: S. Gould et al., Decomposing a Scene into Geometric and Semantically Consistent Regions A. Fischer and C. Igel, An Introducton to Restricted Boltzmann Machines	Video
Module 2: Exact Inference
Monday, Jan 27	Lecture 4 (Eric) - Slides Variable Elimination	Pradeep Dasigi, Soumya Batra, Manzil Zaheer (Scribe Notes)	Required: Jordan Textbook, Ch. 3 Optional: Koller and Friedman Textbook, Ch. 9	Video
Wednesday, Jan 29	Lecture 5 (Eric) - Slides Sum-Product Message Passing	Zhe Zhang, Alex Loewi (Scribe Notes)	Required: Jordan Textbook, Ch. 4 Optional: Koller and Friedman Textbook, Ch. 10	Video
Module 3: Learning
Monday, Feb 3	Lecture 6 (Eric) - Slides Generalized Linear Models Maximum Likelihood Estimation Sufficient Statistics	Alnur Ali, Yipei Wang (Scribe Notes)	Required: Jordan Textbook, Ch. 8	Video
Wednesday, Feb 5	Lecture 7 (Eric) - Slides Learning in Fully Observed Bayesian Networks	Diyi Yang, Zichao Yang (Scribe Notes)	Required: Jordan Textbook, Ch. 9, Sec. 9.1-9.2 Optional: Koller and Friedman Textbook, Ch. 17	Video
Monday, Feb 10	Lecture 8 (Eric) - Slides Learning in Fully Observed Markov Networks	Li Zhou, Meng Song (Scribe Notes)	Required: Jordan Textbook, Ch. 9, Sec. 9.3-9.5 Optional: J. Lee and T. Hastie, Structure Learning of Mixed Graphical Models H. Wallach, Conditional Random Fields: An Introduction	Video
Wednesday, Feb 12	Lecture 9 (Eric) - Slides The Expectation-Maximization Algorithm	Rohan Ramanath, Rahul Goutam (Scribe Notes)	Required: Jordan Textbook, Ch. 10 Optional: A. Dempster et al., Maximum Likelihood from Incomplete Data via the EM Algorithm	Video
Module 4: Popular Graphical Models
Monday, Feb 17	Lecture 10 (Eric) - Slides Modeling Networks Ising Models Gaussian Graphical Models	Zhiding Yu, Shanghang Zhang (Scribe Notes)	Required: J. Friedman et al., Sparse Inverse Covariance Estimation with the Graphical Lasso Optional: A. Dempster, Covariance Selection N. Meinshausen and P. Buhlmann, High-Dimensional Graphs and Variable Selection with the Lasso	Video
Wednesday, Feb 19	Lecture 11 (Eric) - Slides Factor Analysis State Space Models	Swabha Swayamdipta, Dallas Card (Scribe Notes)	Required: Jordan Textbook, Ch. 14 Jordan Textbook, Ch. 15 Optional: A. Ng, Factor Analysis Lecture Notes G. Welch and G. Bishop, An Introduction to the Kalman Filter	Video
Friday, Feb 21	Lecture 12 (Eric) - Slides Conditional Random Fields Case Study: Image Segmentation in Computer Vision	Qin Gao, Siheng Chen (Scribe Notes)	Required: H. Wallach, Conditional Random Fields: An Introduction F. Sha and F. Pereira, Shallow Parsing with Conditional Random Fields Optional: J. Lafferty et al., Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data S. Kumar and M. Hebert, Discriminative Random Fields	Video
Module 5: Approximate Inference
Monday, Feb 24	Lecture 13 (Eric) - Slides Variational Inference: Loopy Belief Propagation	Rajarshi Das, Zhengzhong Liu, Dishan Gupta (Scribe Notes)	Required: None Optional: Koller and Friedman Textbook, Ch. 11 M. Jordan et al., An Introduction to Variational Inference for Graphical Models M. Wainwright and M. Jordan, Graphical Models, Exponential Families, and Variational Inference, Sec. 4.1	Video
Monday, Mar 3	Lecture 14 (Eric) - Slides Variational Inference: Mean Field Approximation	Amos Ng, Yu-Hsin Kuo (Scribe Notes)	Required: M. Wainwright and M. Jordan, Graphical Models, Exponential Families, and Variational Inference, Sec. 2.6 and Sec. 5 Optional: E. Xing et al., A Generalized Mean Field Algorithm for Variational Inference in Exponential Families	Video
Wednesday, Mar 5	Lecture 15 (Eric) - Slides Variational Inference: Theory Case Study: Learning Topic Models	Jingwei Shen, Jiwei Li (Scribe Notes)	Required: M. Wainwright and M. Jordan, Variational Inference in Graphical Models: The View from the Marginal Polytope D. Blei et al., Latent Dirichlet Allocation Optional: M. Wainwright and M. Jordan, Graphical Models, Exponential Families, and Variational Inference, Sec. 3 and Sec. 4 A. Ahmed and E. P. Xing, On Tight Approximate Inference of Logistic-Normal Admixture Model.	Video
Monday, Mar 10	No Lecture due to CMU spring break.
Wednesday, Mar 12	No Lecture due to CMU spring break.
Monday, Mar 17	Lecture 16 (Eric) - Slides Approximate Inference: Monte Carlo Methods	Calvin Murdock, Veeru Sadhanala, Luis Tandalla (Scribe Notes)	Required: Mackay Textbook, Ch. 29.1 - 29.3 Jordan Textbook, Ch. 21 Optional: D. Mackay, Introduction to Monte Carlo Methods	Video
Friday, Mar 21	Lecture 17 (Eric) - Slides Approximate Inference: Markov Chain Monte Carlo (MCMC) Case Study: Learning Topic Models	Karanhaar Singh, Dan Schwartz, Felipe Hernandez (Scribe Notes)	Required: Mackay Textbook, Chpt 29.4 - 29.10 T. Griffiths and M. Steyvers, Finding Scientific Topics Optional: D. Mackay, Introduction to Monte Carlo Methods	Video
Monday, Mar 24	Lecture 18 (Eric) - Slides Advanced Topics in MCMC	Jessica Chemali, Seungwhan Moon (Scribe Notes)	Required: D. Hastie and P. Green, Model Choice using Reversible Jump Markov Chain Monte Carlo A. Dubey, S. Williamson and E. Xing, Parallel Markov Chain Monte Carlo for Nonparametric Mixture Models Optional: A. Doucet and A. Johansen, A Tutorial on Particle Filtering and Smoothing: Fifteen Years Later, Ch. 3	Video
Module 6: Nonparametric Bayesian Models
Wednesday, Mar 26	Lecture 19 (Eric) - Slides Dirichlet Processes	Carl Malings, Jingkun Gao (Scribe Notes)	Required: Y. W. Teh, Dirichlet Process T. Ferguson, A Bayesian Analysis of Some Nonparametric Problems J. Sethuraman, A Constructive Definition of Dirichlet Priors Optional: C. Rasmussen, Gaussian Processes for Machine Learning	Video
Monday, Mar 31	Lecture 20 (Eric) - Slides Hierarchical Dirichlet Processes Case Study: Genetic Inference of the World Population	Lavanya Viswanathan, Manaal Faruqui (Scribe Notes)	Required: K. A. Sohn and E. Xing, A Hierarchical Dirichlet Process Mixture Model for Haplotype Reconstruction from Multi-Population Data Optional: Y. W. Teh, M. Jordan, M. Beal, and D. Blei, Hierarchical Dirichlet Processes E. Xing, R. Sharan, and M. Jordan, Bayesian Haplotype Inference via the Dirichlet Process	Video
Wednesday, Apr 2	Lecture 21 (Eric) - Slides The Indian Buffet Process	Jesse Dodge, Nijith Jacob (Scribe Notes)	Required: T. Griffiths and Z. Ghahramani, The Indian Buffet Process: An Introduction and Review	Video
Module 7: Spectral Methods for Graphical Models
Monday, Apr 7	Lecture 22 (Eric) - Slides Hilbert Space Embeddings of Distributions	Sujay Kumar Jauhar, Zhiguang Huo (Scribe Notes)	Required: L. Song, Learning via Hilbert Space Embeddings of Distributions, Ch. 2.1-2.2 and Ch. 3.1-3.2	Video
Wednesday, Apr 9	Lecture 23 (Eric) - Slides Kernel Graphical Models	Xiang Li, Ran Chen (Scribe Notes)	Required: L. Song, J. Huang, A. Smola, and K. Fukumizu, Hilbert Space Embeddings of Conditional Distributions with Applications to Dynamical Systems Optional: L. Song, A. Gretton, and C. Guestrin, Nonparametric Tree Graphical Models via Kernel Embeddings L. Song, A. Gretton, D. Bickson, Y. Low, and C. Guestrin, Kernel Belief Propagation	Video
Monday, Apr 14	Lecture 24 (Eric) - Slides Spectral Algorithms for Graphical Models Case Study: Parsing in Natural Languages	Yuan Xie, Yulong Pei, Junier Oliva (Scribe Notes)	Required: D. Hsu, S. Kakade, and T. Zhang A Spectral Algorithm for Learning Hidden Markov Models Optional: L. Song et al., Nonparametric Latent Tree Graphical Models: Inference, Estimation, and Structure Learning A. Parikh, L. Song, and E. Xing, A Spectral Algorithm for Latent Tree Graphical Models L. Song et al., Hilbert Space Embeddings of Hidden Markov Models L. Song et al., Kernel Embeddings of Latent Tree Graphical Models S. Cohen et al., Spectral Learning of Latent-Variable PCFGs	Video
Module 8: Structured Sparsity
Wednesday, Apr 16	Lecture 25 (Eric) - Slides Graph-Induced Structured Input-Output Methods Case Study: Disease Association Analysis	Alok Kothari, Haoyu Wang (Scribe Notes)	Required: S. Kim and E. Xing, Statistical Estimation of Correlated Genome Associations to a Quantitative Trait Network X. Chen et al., Smoothing Proximal Gradient Method for General Structured Sparse Regression Optional: S. Kim and E. Xing, Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity	Video
Monday, Apr 21	Lecture 26 (Eric) - Slides Structured Sparse Additive Models	Ruikun Luo, Hao Zhang (Scribe Notes)	Required: P. Ravikumar, J. Lafferty, H. Liu, and L. Wasserman, Sparse Additive Models J. Yin, X. Chen, and E. Xing, Group Sparse Additive Models Optional: T. Hastie and R. Tibshirani, Generalized Additive Models	Video
Module 9: Scalable Algorithms for Graphical Models
Wednesday, Apr 23	Lecture 27 (Qirong, Avi) - Slides Distributed MCMC	Pengtao Xie, Khoa Luu (Scribe Notes)	Required: J. Gonzalez et al., Parallel Gibbs Sampling: From Colored Fields to Thin Junction Trees Optional: A. Doucet and A. Johansen, A Tutorial on Particle Filtering and Smoothing: Fifteen Years Later S. Williamson, A. Dubey, and E. Xing, Parallel Markov Chain Monte Carlo for Nonparametric Mixture Models	Video
Module 10: Posterior Regularization and Max-Margin Graphical Models
Monday, Apr 28	Lecture 28 (Eric) - Slides Maximum-Margin Learning of Graphical Models	T. H. Ngan Le, Felipe Hernandez (Scribe Notes)	Required: B. Taskar, C. Guestrin, and D. Koller, Max-Margin Markov Networks J. Zhu and E. Xing, Maximum Entropy Discrimination Markov Networks Optional: J. Zhu and E. Xing, On Primal and Dual Sparsity of Markov Networks J. Zhu, E. Xing, and B. Zhang, Partially Observed Maximum Entropy Discrimination Markov Networks	Video
Wednesday, Apr 30	Lecture 29 (Eric) - Slides Posterior Regularization: An Integrative Paradigm for Learning Graphical Models	Felix Juefei Xu, Abhishek Chugh (Scribe Notes)	Required: J. Zhu, A. Ahmed, and E. Xing, MedLDA: Maximum Margin Supervised Topic Models for Regression and Classification J. Zhu, N. Chen, and E. Xing, Bayesian Inference with Posterior Regularization and Applications to Infinite Latent SVMs	Video