10-708 – Lectures (tentative)
2020 Spring
| Lecture | Date | Topic | Slides | Videos | Further Reading | Note | Scribe
| | Design of GMs |
| 01 | Jan 13 | Introduction to GM: (Eric)
- Association between random variables
- Marginal/partial correlation
- Conditional independence | pdf | panopto,
youtube | Jordan 2004, Airoldi 2007,
Larry's notes | | pdf |
| 02 | Jan 15 | Undirected GMs: (Eric)
- Markov property | pdf | panopto,
youtube | Koller & Friedman Ch. 4 | | pdf |
| Jan 20 | | | | | MLK day | |
| 03 | Jan 22 | Directed GMs: (Eric)
- Markov property | pdf | panopto,
youtube | Koller & Friedman Ch. 3 | hw1 out | pdf
| | Basic Inference and Learning |
| 04 | Jan 27 | Exact Inference: (Eric)
- Variable elimination
- Sum-product on trees
- Belief propagation on junction trees | pdf | panopto,
youtube | Jordan Ch. 3, Ch. 4,
Koller & Friedman Ch. 9, Ch. 10 | | pdf |
| 05 | Jan 29 | Parameter Estimation: (Eric)
- Fully observed: MLE, MAP, Bayesian
- Exponential family distributions, GLMs
- Partially observed: EM algorithm | pdf | panopto,
youtube | Koller & Friedman Ch. 17.1-4,
Ch. 19.1-4,
Neal & Hinton 1998 | | pdf |
| 06 | Feb 03 | Case Studies: HMM and CRF (Eric) | pdf | panopto,
youtube | Koller & Friedman Ch. 6.2,
Wallach 2004,
Lafferty, McCallum, Pereira 2001 | | pdf
| | Approximate Inference |
| 07 | Feb 05 | Variational Inference 1: (Eric)
- Variational methods
- LDA | pdf | panopto,
youtube | Wainwright, Jordan 2008 | | pdf |
| 08 | Feb 10 | Variational Inference 2: (Xun)
- Stochastic/Black-box VI
- VI Theory | pdf | panopto,
youtube | Wainwright, Jordan 2008 | | pdf |
| 09 | Feb 12 | Sampling 1: (Eric)
- Accept-reject sampling
- Importance sampling
- Metropolis-Hastings
- Gibbs sampling | pdf | panopto,
youtube | MacKay 2003, Ch. 29.1-3 | hw1 due | pdf |
| 10 | Feb 17 | Sampling 2: (Eric)
- Hamiltonian Monte Carlo
- Langevin dynamics
- Sequential Monte Carlo | pdf | panopto,
youtube | MacKay 2003, Ch. 29.4-10 | | pdf
| | Deep Learning and Deep Generative Models |
| 11 | Feb 19 | Foundations of Deep Learning: (Eric)
- Insight into DL
- Connectionss to GM | pdf | panopto,
youtube | Goodfellow, Bengio, Courville 2016
Ch. 6.2-5, 20.3-4 | proposal due, hw2 out | pdf |
| 12 | Feb 24 | Deep Generative Models 1: (Eric)
- Wake-sleep algorithm
- Variational autoencoder
- Generative adversarial networks | pdf | panopto,
youtube | Goodfellow, Bengio, Courville 2016
Ch. 20.9-10
Mohamed et al. 2019 | | pdf |
| 13 | Feb 26 | Deep Generative Models 2: (Eric)
- More GANs and variants
- Normalizing flows
- Integrating domain knowledge in DL | pdf | panopto,
youtube | Arjovsky, Bottou 2017,
Papamakarios et al. 2019,
Hu et al. 2018 | | pdf |
| 14 | Mar 02 | Deep Sequence Models: (Zhiting)
- RNN and LSTM
- CNN and Transformers
- Attention mechanisms | pdf | panopto,
youtube | Pascanu, Mikolov, Bengio 2013,
Vaswani et al. 2017,
Devlin et al. 2018 | | pdf |
| 15 | Mar 04 | Case Study: Text Generation (Zhiting)
- Encoder-decoder framework
- Machine translation as conditional generation
- Unifying MLE and RL for text generation | pdf | panopto,
youtube | Ranzato et al. 2015,
Hu et al. 2017 | hw2 due, hw3 out | pdf |
| Mar 09 | | | | | Spring break | |
| Mar 11 | | | | | Spring break |
| | Structure and Causal Inference |
| Mar 16 | | | | | No class. Stay healthy! | |
| 16 | Mar 18 | Structure Learning (Eric):
- Undirected GM: Gaussian GM
- Directed: Causal discovery | pdf | panopto,
youtube | Meinshausen, Bühlmann 2006,
Kolar et al. 2010 | | pdf |
| 17 | Mar 23 | Causality 1: (Kun Zhang) | pdf | panopto,
youtube | Pearl et al. 2016 | | pdf |
| 18 | Mar 25 | Causality 2: (Kun Zhang) | pdf | panopto,
youtube | Spirtes et al. 1993,
Zhang et al. 2017 | | pdf
| | RL as Inference in GMs |
| 19 | Mar 30 | RL as Inference 1 (Maruan) | pdf | panopto,
youtube | Sutton, Barto Ch. 3-4,
Lilian Weng blog post,
Levine Sec. 1-4,
Ziebart Ch. 5.1-2, 6.1-2 | | pdf |
| 20 | Apr 01 | RL as Inference 2 (Maruan) | pdf | panopto,
youtube | | hw3 due, hw4 out | pdf
| | RL as Inference in GMs |
| 21 | Apr 06 | Gaussian Process (Eric) | pdf | panopto,
youtube | | | pdf |
| 22 | Apr 08 | Determinantal Point Process (Pengtao) | | panopto,
youtube | | midway report due | pdf
| | Bayesian Nonparametrics |
| 23 | Apr 13 | Dirichlet Process (Eric) | pdf | panopto,
youtube | | | pdf |
| 24 | Apr 15 | Indian Buffet Process (Eric) | pdf | panopto,
youtube | | | pdf
| | Applications and Systems |
| 25 | Apr 20 | Spectral Graphical Models (Eric) | pdf | panopto,
youtube | | | |
| 26 | Apr 22 | Large-scale Algorithms and Systems (Qirong) | pdf | panopto,
youtube | | | |
| 27 | Apr 27 | Meta-Learning (Maruan) | pdf | panopto,
youtube | | | |
| 28 | Apr 29 | Robust Machine Learning (Haohan) | pdf | panopto,
youtube | | final report due |
|
Video playlists: Panopto, Youtube
Candidates for open slots:
Structure Learning for Markov Networks
Theory of Variational Inference
Spectral and Kernel GMs
Max-margin GMs
Regularized Bayesian Learning
Meta-learning and Neural Process
...
|