Title: Syllabus
Course: F19 Introduction to Deep Learning

Tentative Timeline

6-Dec-19 | Recitation 00 | Foundations of Python, NumPy, Jupyter Notebook
4-Dec-19 | Recitation 01 | Amazon Web Services (AWS)
2-Dec-19 | Recitation 02 | Your First Deep Learning Code
29-Nov-19 | Recitation 03 | Efficient Deep Learning/Optimization Methods
29-Nov-19 | Recitation 04 | Debugging and Visualization
27-Nov-19 | Recitation 05 | Convolutional Neural Networks
25-Nov-19 | Recitation 06 | CNNs: HW2
22-Nov-19 | Recitation 07 | Recurrent Neural Networks
20-Nov-19 | Recitation 08 | RNN: CTC
18-Nov-19 | Recitation 09 | Attention
15-Nov-19 | Recitation 10 | Variation Auto Encoders
15-Nov-19 | Recitation 11 | GANs
13-Nov-19 | Recitation 12 | RL
11-Nov-19 | Recitation 13 | Boltzmann machines
8-Nov-19
Quiz 01
Syllabus
History and cognitive basis of neural computation.
The perceptron / multi-layer perceptron
The neural net as a universal approximator
8-Nov-19
Quiz 02
Training a neural network
Perceptron learning rule
Empirical Risk Minimization
Optimization by gradient descent
6-Nov-19 | Quiz 03 | Back propagation Calculus of back propogation Back propagation Continued
4-Nov-19
Quiz 04
Convergence in neural networks
Rates of convergence
Loss surfaces
Learning rates, and optimization methods
RMSProp, Adagrad, Momentum
Cognitive and Brain Science
Neural Basis of Cognition
1-Nov-19
Quiz 05
Stochastic gradient descent
Acceleration
Overfitting and regularization
Tricks of the trade:
Choosing a divergence (loss) function
Batch normalization
Dropout
Convolutional Neural Networks (CNNs)
Weights as templates
Translation invariance
Training with shared parameters
Arriving at the convlutional model
1-Nov-19
Quiz 06
Models of vision
Neocognitron
Mathematical details of CNNs
Alexnet, Inception, VGG
Recurrent Neural Networks (RNNs)
Modeling series
Back propogation through time
Bidirectional RNNs
30-Oct-19
Quiz 07
Stability
Exploding/vanishing gradients
Long Short-Term Memory Units (LSTMs) and variants
Resnets
Loss functions for recurrent networks
Sequence prediction
25-Oct-19 | Quiz 08 | Sequence To Sequence Methods Connectionist Temporal Classification (CTC) Sequence-to-sequence models Attention models examples from speech and language
25-Oct-19
Quiz 09
What to networks represent
Autoencoders and dimensionality reduction
Learning representations
Variational Autoencoders (VAEs)
23-Oct-19 | Quiz 10 | Generative Adversarial Networks (GANs) Part 1 Generative Adversarial Networks (GANs) Part 2
21-Oct-19
Quiz 11
Hopfield Networks
Boltzmann Machines
Training Hopfield Networks
Stochastic Hopfield Networks
18-Oct-19 | Quiz 12 | Restricted Boltzman Machines Deep Boltzman Machines Reinforcement Learning 1
18-Oct-19 | Quiz 13 | Reinforcement Learning 2 Reinforcement Learning 3
16-Oct-19 | Quiz 14 | Reinforcement Learning 4
14-Oct-19
Quiz 15
Q Learning
Deep Q Learning
Newer models and trends
Review
11-Oct-19 | Lecture 01 | Introduction to deep learning History and cognitive basis of neural computation. The perceptron / multi-layer perceptron
11-Oct-19 | Lecture 02 | The neural net as a universal approximator
9-Oct-19
Lecture 03
Training a neural network
Perceptron learning rule
Empirical Risk Minimization
Optimization by gradient descent
7-Oct-19 | Lecture 04 | Back propagation Calculus of back propogation
4-Oct-19 | Lecture 05 | Back propagation Continued
4-Oct-19 | Lecture 06 | Cognitive and Brain Science Neural Basis of Cognition
2-Oct-19
Lecture 07
Convergence in neural networks
Rates of convergence
Loss surfaces
Learning rates, and optimization methods
RMSProp, Adagrad, Momentum
30-Sep-19
Lecture 08
Stochastic gradient descent
Acceleration
Overfitting and regularization
Tricks of the trade:
Choosing a divergence (loss) function
Batch normalization
Dropout
27-Sep-19
Lecture 09
Convolutional Neural Networks (CNNs)
Weights as templates
Translation invariance
Training with shared parameters
Arriving at the convlutional model
27-Sep-19
Lecture 10
Models of vision
Neocognitron
Mathematical details of CNNs
Alexnet, Inception, VGG
25-Sep-19
Lecture 11
Recurrent Neural Networks (RNNs)
Modeling series
Back propogation through time
Bidirectional RNNs
23-Sep-19
Lecture 12
Stability
Exploding/vanishing gradients
Long Short-Term Memory Units (LSTMs) and variants
Resnets
20-Sep-19 | Lecture 13 | Loss functions for recurrent networks Sequence prediction
20-Sep-19 | Lecture 14 | Sequence To Sequence Methods Connectionist Temporal Classification (CTC)
18-Sep-19 | Lecture 15 | Sequence-to-sequence models Attention models examples from speech and language
16-Sep-19 | Lecture 16 | What to networks represent Autoencoders and dimensionality reduction Learning representations
13-Sep-19 | Lecture 17 | Variational Autoencoders (VAEs)
13-Sep-19 | Lecture 18 | Generative Adversarial Networks (GANs) Part 1
11-Sep-19 | Lecture 19 | Generative Adversarial Networks (GANs) Part 2
9-Sep-19 | Lecture 20 | Hopfield Networks Boltzmann Machines
6-Sep-19 | Lecture 21 | Training Hopfield Networks Stochastic Hopfield Networks
6-Sep-19 | Lecture 22 | Restricted Boltzman Machines Deep Boltzman Machines
4-Sep-19 | Lecture 23 | Reinforcement Learning 1
30-Aug-19 | Lecture 24 | Reinforcement Learning 2
30-Aug-19 | Lecture 25 | Reinforcement Learning 3
28-Aug-19 | Lecture 26 | Reinforcement Learning 4
26-Aug-19 | Lecture 27 | Thanksgiving break, no classes
15-Aug-19 | Lecture 28 | Q Learning Deep Q Learning
28-Oct-29 | Lecture 29 | Newer models and trends Review