Generative AI

• Sequence Modeling: Recurrent and Recursive Nets. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.1-10.5.
• A Framework for the Cooperation of Learning Algorithms. Léon Bottou, Patrick Gallinari (1991). Advances in Neural Information Processing Systems.
• PyTorch: An Imperative Style, High-Performance Deep Learning Library. Paszke et al. (2019). Advances in Neural Information Processing Systems.

HW0 out

• The Long Short-Term Memory and Other GatedRNNs. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.10-10.12.
• Attention Is All You Need. Vaswani et al. (2017). NeurIPS.
• The Illustrated Transformer. Alammar (2018).

• Generating Sequences With Recurrent Neural Networks. Graves (2014).
• Recurrent neural network based language model. Mikolov et al. (2010). Interspeech.
• [GPT-1] Improving Language Understanding by Generative Pre-Training. Radford et al. (2018).
• [GPT-2] Language Models are Unsupervised Multitask Learners. Radford et al. (2019).

HW0 due

• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. Ainslie et al. (2023). EMNLP.
• Longformer: The Long-Document Transformer. Beltagy et al. (2020).
• RoFormer: Enhanced Transformer with Rotary Position Embedding. Su et al. (2021).
• Convolutional Networks. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 9.1 - 9.3.

Add/audit/tuition drop deadline

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Devlin et al. (2018). NAACL.
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Dosovitskiy et al. (2021). ICLR.

Quiz 1 (in-class, L1-L3)

• Generative Adversarial Nets. Goodfellow et al. (2014). NeurIPS.
• NuerIPS 2016 Tutorial: Generative Adversarial Networks. Goodfellow (2017). NeurIPS Tutorials.

• Deep Unsupervised Learning using Nonequilibrium Thermodynamics. Sohl-Dickstein et al. (2015). MLR.
• Denoising Diffusion Probabilistic Models. Ho et al. (2020). NeurIPS.

• Variational Inference: A Review for Statisticians. Blei, Kucukelbir, and McAuliffe (2018).
• High-Level Explanation of Variational Inference. Jason Eisner (2011).
• Tutorial on Variational Autoencoders. Carl Doersch (2016).

Quiz 2 (in-class, L4-L8)

HW2 due

HW3 out (L8-L11)

(Quiz 3 in-class, L9-L11)

Semester Course Drop Deadline

Send out course Survey

HW3 due

Midsemester Grades Due

HW4 out (L12-L15)

(Quiz 4 in-class, L12-L15)

HW4 due

HW623 out

Project team formation due by 2pm

HW623 due

(Quiz 5 in-class, L16-L20)

Registration week

Project midway report due

Project poster due

Project final report due