Lecture Schedule

Lectures are held on Mondays and Wednesdays from 12:00-1:20 pm in DH 1212.

Date	Lecture	Scribes	Readings	Anouncements
Monday, Jan 12	Lecture 1 (Eric): Introduction to GM - Slides	Wenbo Liu, Venkata Krishna Pillutla Notes	Required (no reading summary): Jordan Textbook, Ch 2 (will distribute shortly) Optional: Koller and Friedman Textbook, Ch. 3
Module 1: Representation
Wednesday, Jan 14	Lecture 2 (Eric): Directed GMs: Bayesian Networks - Slides	Yi Cheng, Cong Lu Notes	Required (please bring your reading summary): Jordan Textbook, Ch 2 (Section 2.2 - end) Optional: Koller and Friedman Textbook, Ch. 4
Monday, Jan 19	No Lecture due to MLK day.
Wednesday, Jan 21	Lecture 3 (Eric): Representation of Undirected GM - Slides, Annotated	Karima Ma, Manu Reddy Nannuri Notes	Required (please bring your reading summary): A. Fischer and C. Igel, An Introducton to Restricted Boltzmann Machines Optional: S. Gould et al., Decomposing a Scene into Geometric and Semantically Consistent Regions C. K. Chow and C. N. Liu, Approximating Discrete Probability Distributions with Dependency Trees B. A. Cipra, An Introduction to the Ising Model
Module 2: Inference and Learning
Monday, Jan 26	Lecture 4 (Eric): - Slides, Annotated Maximum likelihood parameter estimation Bayesian inference Regularization	How Jing, Xiaoqiu Huang Notes	Required (please bring your reading summary): Jordan Textbook, Ch. 8 Optional: T. Park and G. Casella, The Bayesian Lasso R. Tibshirani, Regression Shrinkage and Selection via the Lasso
Tuesday, Jan 27	Lecture 5 (Eric): - Slides, Annotated Generalized linear model and sufficient statistics Learning fully observed directed GMs	Uttara Ananthakrishnan, Lujie(Karen) Chen, Mallory Nobles Notes	Required (please bring your reading summary): Jordan Textbook, Ch. 9, Sec. 9.1-9.2 Optional: Koller and Friedman Textbook, Ch. 17	Assignment 1 is out. Due on Feb 13 at 12 noon
Monday, Feb 2	Lecture 6 (Yaoliang): Learning fully observed undirected GM. - Slides	Satwik Kottur, William Herlands, Maria De Arteaga Notes	Required (please bring your reading summary): Jordan Textbook, Ch. 9, Sec. 9.3-9.5 Optional: J. Lee and T. Hastie, Structure Learning of Mixed Graphical Models H. Wallach, Conditional Random Fields: An Introduction
Wednesday, Feb 4	Lecture 7 (Seunghak): Exact Inference: - Slides Elimination and Message Passing The Sum Product Algorithm	Vipul Singh, Xiao Liu, Vrushali Fangal Notes	Required (please bring your reading summary): Jordan Textbook, Ch. 3 Jordan Textbook, Ch. 4 Optional: Koller and Friedman Textbook, Ch. 9 Koller and Friedman Textbook, Ch. 10
Monday, Feb 9	Lecture 8 (Eric): Learning Partially observed models: - Slides, Annotated The EM algorithm	Aurick Qiao, Hao Zhang, Bing Liu Notes	Required (please bring your reading summary): Jordan Textbook, Ch. 10 Optional: A. Dempster et al., Maximum Likelihood from Incomplete Data via the EM Algorithm
Module 3: Popular Graphical Models
Wednesday, Feb 11	Lecture 9 (Bin Zhao): Case Study with Popular GMs I: - Slides HMM vs CRF CRFs for computer vision	Emmanouil Antonios Platanios, Mariya Toneva, Jeya Balaji Balasubramanian Notes	Required (please bring your reading summary): J. Lafferty et al., Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data F. Sha and F. Pereira, Shallow Parsing with Conditional Random Fields Optional: S. Kumar and M. Hebert. Discriminative Random Fields. IJCV, 2006 Xuming He, Richard Zemel, and Miguel Carreira-Perpinan. Multiscale conditional random fields for image labeling. CVPR 2004
Monday, Feb 16	Lecture 10 (Eric): - Slides, Annotated Multivariate Gaussian Gene network	Yan Xia, Dexter Min Hyung Lee Notes	Required (please bring your reading summary): J. Friedman et al., Sparse Inverse Covariance Estimation with the Graphical Lasso M. Kolar et al., Estimating Time-Varying Networks Optional: A. Dempster, Covariance Selection N. Meinshausen and P. Buhlmann, High-Dimensional Graphs and Variable Selection with the Lasso M. Kolar et al., Graph Estimation From Multi-attribute Data
Wednesday, Feb 18	Lecture 11 (Eric): - Slides, Annotated Factor Analysis State Space Models Topic/Trend Tracking	Yilin He, Udbhav Prasad Notes	Required (please bring your reading summary): Jordan Textbook, Ch. 14 Jordan Textbook, Ch. 15 Optional: A. Ng, Factor Analysis Lecture Notes G. Welch and G. Bishop, An Introduction to the Kalman Filter	Project proposal due at noon; Assignment 2 is out
Module 4: Approximate Inference
Monday, Feb 23	Lecture 12 (Eric): Variational inference I: - Slides, Annotated Loopy Belief Propagation	Evan Shapiro, Eric Lei, Fattaneh Jabbari Notes	Required (please bring your reading summary): Yedidia et al., Generalized Belief Propagation Wainwright and Jordan, Graphical Models, Exponential Families, and Variational Inference, Sec. 4.1 Optional: Murphy et al., Loopy Belief Propagation for Approximate Inference: An Empirical Study
Wednesday, Feb 25	Lecture 13 (Willie): Variational inference II: - Slides Mean field	Yuntian Deng, Zhiting Hu, Ronghuo Zheng Notes	Required (please bring your reading summary): Xing et al., A Generalized Mean Field Algorithm for Variational Inference in Exponential Families Wainwright and Jordan, Graphical Models, Exponential Families, and Variational Inference, Sec. 5.1-5.3
Monday, Mar 2	Lecture 14 (Eric): Theory of variational inference - Slides, Annotated	Abhinav Maurya, Joey Robinson, Qian Wan Notes	Required (please bring your reading summary): M. Wainwright and M. Jordan, Variational Inference in Graphical Models: The View from the Marginal Polytope Optional: M. Wainwright and M. Jordan, Graphical Models, Exponential Families, and Variational Inference, Sec. 3 and Sec. 4	Assignment 2 due
Wednesday, Mar 4	Lecture 15 (Eric): Case study with Popular GMs II: - Slides Topic models	Xinyu Miao, Yun Ni, Linglin Huang Notes	Required (please bring your reading summary): D. Blei et al., Latent Dirichlet Allocation Optional: T. Griffiths and M. Steyvers, Finding Scientific Topics A. Ahmed and E. P. Xing, On Tight Approximate Inference of Logistic-Normal Admixture Model.
Monday, Mar 9	No Lecture due to CMU spring break.
Wednesday, Mar 11	No Lecture due to CMU spring break.
Monday, Mar 16	Lecture 16 (Eric): - Slides, Annotated Monte Carlo: Basic concept Importance sampling Particle filtering Sequential Monte Carlo	Jonathan deWerd, Jay Yoon Lee, Aaron Li Notes	Required: Mackay Textbook, Ch. 29.1 - 29.3 Jordan Textbook, Ch. 21 Optional: D. Mackay, Introduction to Monte Carlo Methods
Wednesday, Mar 18	Lecture 17 (Andrew): MCMC - Slides Pitfalls of Monte Carlo Markov chains Metropolis Hastings Gibbs Sampling Slice Sampling Hamiltonian Monte Carlo Simulated Annealing and Parallel Tempering	Heran Lin, Bin Deng, Yun Huang Notes	Required: Iain Murray's PhD thesis, pp. 20-55 Optional: Mackay Textbook, Ch. 29, 30. C. Bishop, Pattern Recognition and Machine Learning (PRML), Ch. 11 R. Neal, Slice Sampling, Annals of Statistics, 2003 C. Geyer, Practical Markov chain Monte Carlo, Statistical Science 7(4): 473-492. 1992. J. Geweke, Getting it right: joint distribution tests of posterior simulators, JASA 99(467): 799-804, 2004.
Module 5: Nonparametric Bayesian Models
Monday, Mar 23	Lecture 18 (Avinava): Dirichlet Process and Dirichlet Process Mixtures - Slides	Ji Oh Yoo, Ying Zhang, Chi Liu Notes	Required: Y. W. Teh, Dirichlet Process Optional: Y. W. Teh, Dirichlet Processes: Tutorial and Practical Course, MLSS 2007.
Wednesday, Mar 25	Lecture 19 (Avinava): Indian Buffet Process - Slides	Rishav Das, Adam Brodie, Hemank Lamba Notes	Required: Thomas L. Griffiths and Zoubin Ghahramani, The Indian Buffet Process: An Introduction and Review, JMLR 2011.	Midway report due at 4pm, Mar 26; Assignment 3 is out
Monday, Mar 30	Lecture 20 (Andrew): Gaussian Processes - Slides Probabilistic modelling Bayesian model selection and nonparametric models Linear basis models Gaussian processes and kernels	Haohan Wang, Yuetao Xu, Jisu Kim Notes	Required: C. Rasmussen and C. Williams, GPML, Preface + Ch. 2.2-2.4 C. Rasmussen and Z. Ghahramani, Occam's razor, NIPS 2001. Optional: Wilson PhD Thesis, 2014, Ch. 1, 2. C. Rasmussen and C. Williams, GPML, Ch. 3, 5 D. MacKay, Introduction to Gaussian Processes, 1998.
Wednesday, Apr 1	Lecture 21 (Andrew): Advanced Gaussian Processes - Slides Brief review Kernel construction and derivations Scalable methods	Konstantin Genin, Yutong Zheng Notes	Required: C. Rasmussen and C. Williams, GPML, Ch. 4, 5 J. Candela and C.E. Rasmussen, A unifying view of sparse approximation Gaussian process regression, JMLR 2005. Optional: Lázaro-Gredilla et al., Sparse spectrum Gaussian process regression, JMLR 2011. A. G. Wilson and R. P. Adams, Gaussian process kernels for pattern discovery and extrapolation, ICML 2013.
Module 6: Optimization view of Graphical Models
Monday, Apr 6	Lecture 22 (Yaoliang): Optimization and GMs - Slides Matrix factorization vs LDA	Yu-Xiang Wang, Su Zhou Notes	Required: B. A. Olshausen and D. J. Field, Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature 1996. J. Zhu and E. P. Xing, Sparse Topical Coding, UAI 2011. Optional: B. A. Olshausen and D. J. Field, Sparse coding with an overcomplete basis set: A strategy employed by V1?, Vision Research 1997.
Wednesday, Apr 8	Lecture 23 (Eric): Max-margin learning of GMs - Slides	Xun Zheng, Wei Yu, Lee Gao Notes	Required: B. Taskar, C. Guestrin, and D. Koller, Max-Margin Markov Networks J. Zhu and E. Xing, Maximum Entropy Discrimination Markov Networks
Monday, Apr 13	Lecture 24 (Yaoliang): Regularized Bayesian learning of GMs - Slides	Rose C. Kanjirathinkal, Yiming Gu Notes	Required: J. Zhu, N. Chen, and E. P. Xing, Bayesian Inference with Posterior Regularization and Applications to Infinite Latent SVMs, JMLR 2014. Optional: K. Ganchev, J. Graça, J. Gillenwater, and B. Taskar, Posterior Regularization for Structured Latent Variable Models, JMLR 2010.	Assignment 3 due
Module 7: Advanced Topics
Wednesday, Apr 15	Lecture 25 (Eric): Deep neural networks and GMs - Slides	Harry Gifford, Pradeep Karuturi Notes	Required: L. Deng, A Tutorial Survey of Architectures, Algorithms, and Applications for Deep Learning, Transactions on Signal and Information Processing 2013. Optional: Y. Bengio, Learning Deep Architectures for AI, Foundations and Trends in Machine Learning, 2009. G. Hinton, A Practical Guide to Training Restricted Boltzmann Machines, Tech Report, 2010. M. A. Ranzatto, Y. L. Boureau and Y. LeCun, Sparse Feature Learning for Deep Belief Networks, NIPS, 2007.	Assignment 4 is out
Monday, Apr 20	Lecture 26 (Eric): Spectral GMs - Slides	Guillermo Andres Cidre, Abelino Jimenez Notes	Required: D. Hsu, S. Kakade, and T. Zhang A Spectral Algorithm for Learning Hidden Markov Models Optional: L. Song et al., Nonparametric Latent Tree Graphical Models: Inference, Estimation, and Structure Learning A. Parikh, L. Song, and E. Xing, A Spectral Algorithm for Latent Tree Graphical Models L. Song et al., Hilbert Space Embeddings of Hidden Markov Models L. Song et al., Kernel Embeddings of Latent Tree Graphical Models
Wednesday, Apr 22	Lecture 27 (Eric): Case study with popular GM III: - Slides Graph-induced structured regression for genome association	Elizabeth Silver, Hyun Ah Song Notes	Required: S. Kim and E. Xing, Statistical Estimation of Correlated Genome Associations to a Quantitative Trait Network X. Chen et al., Smoothing Proximal Gradient Method for General Structured Sparse Regression Optional: S. Kim and E. Xing, Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity C. Xiaohui et al., A Two-Graph Guided Multi-task Lasso Approach for eQTL Mapping
Module 8: Scalable Algorithms for Graphical Models
Monday, Apr 27	Lecture 28 (Avinava): Big Learning: - Slides Distributed MCMC	Hakim Sidahmed, Aman Gupta Notes	Required J. Gonzalez et al., Parallel Gibbs Sampling: From Colored Fields to Thin Junction Trees Optional: A. Doucet and A. Johansen, A Tutorial on Particle Filtering and Smoothing: Fifteen Years Later S. Williamson, A. Dubey, and E. Xing, Parallel Markov Chain Monte Carlo for Nonparametric Mixture Models W. Neiswanger, C. Wang, E. Xing, Asymptotically Exact, Embarrassingly Parallel MCMC
Wednesday, Apr 29	Lecture 29 (Yaoliang): Big Learning: - Slides Distributed ADMM for GGM	Taiyuan Zhang, Vrushali Fangal Notes	Required H. Wang et al., Large Scale Distributed Sparse Precision Estimation, NIPS 2013. Optional: T. Cai et al., A constrained l_1 minimization approach to sparse precision matrix estimation, JASA 2011.	Assignment 4 due

 

© 2009 Eric Xing @ School of Computer Science, Carnegie Mellon University
Last updated 04/11/2025 [validate xhtml]