MLSP 11-755/18-797

Date	Lecture Topics (Tentative)	Suggested Readings	Assignments
Aug 28 slides	Lecture 1: Introduction Introduction to course Notion of a signal Basic digital representation of data E.g. speech, images, other types of data	Foundations of Signal Processing. Ch 1.
Aug 30 slides	Lecture 2: Basics. Fundamentals of Linear Algebra, part 1 Vector Spaces Algebraic operations and their interpretations Projections	Linear Algebra Ch 1.3 Deep Learning Book. Ch 2.1-2.6	Quiz 1 Due Sept 2
Sept 6 slides	Lecture 3: Basics. Fundamentals of Linear Algebra, part 2 Basics of Calculus Types of operations Eigen decomposition, SVD Vector and Matrix Calculus Function Spaces	Linear Algebra Ch 6.1 Linear Algebra Ch 7.1 The Matrix Cookbook. Ch 2. Deep Learning Book. Ch 2.7-2.11	Quiz 2 Due Sept 9
Sept 11 slides	Lecture 4: Representations Projects ideas discussion The idea behind deterministic representations Wavelets Fourier Transform Cosine Transforms	DFT Wavelets and Signal Processing Time frequency Representation
Sept 13 slides	Lecture 5: Optimization Gradient ascent/descent Basics of convex optimization Constrained optimization Lagrange multipliers Projected gradients	Convex Optimization Ch 9	Quiz 3 Due Sept 16 HW 1 out
Sept 18 slides	Lecture 6: Representations Data-driven representations Eigen representations Karhunen-Loeve PCA Properties	Eigenfaces for Recognition Tutorial on PCA
Sept 20 slides	Lecture 7: Classification Introduction to Classification Binary Classification Boosting Application to Face Detection	Statistical Learning. Ch 10	Quiz 4 Due Sept 23
Sept 25 slides	Lecture 8: Classification Face Detection Integral Image Cascade Classifier Pratical Implementation	Viola-Jone's paper
Sept 27 slides	Lecture 9: Representations Data-driven representations Independent Component Analysis ICA for representations and denoising ICA applications	ICA, new concept? ICA, algorithms and applications	Quiz 5 Due Sept 30 Project Proposal Due Sept 30 HW1 submission Due Oct 1st
Oct 2 slides	Lecture 9 (part 2): Representations Data-driven representations Independent Component Analysis Information Theoretical based Methods Applications	Spectrogram dimensionality reduction with independence constraints
Oct 4 slides	Lecture 10: Representations Data-driven representations Non-negative matrix factorization Types of NMF Overcomplete representations Sparsity Applications	Algorithms for NNMF Speech Denoising using NNMF with prios	HW 2 out Quiz 6 Due Oct 7
Oct 9 slides	Lecture 11: Modelling/Representations Clustering Basic idea K-means Bag of Words Kernels and Mercer's condition Kernel K-means	Spectral Clustering Efficient ASD prediction with eye movement
Oct 11 slides	Lecture 12: Representations/Modelling Dictionary based representations Sparse and overcomplete representations Application to denoising	CoSAMP Sparse Modeling of Signals and Images	Quiz 7 Due Oct 14
Oct 16 slides	Lecture 13: Prediction and Modeling Nearest neighbors Linear regression, Kernel regression, Regularization, Tikhonov and L1 regularization Sparsity	Locality Sensitive Hashing Random Features for Large Scale Kernel Machines
Oct 18 slides	Lecture 14: Classification Linear classifiers Perceptrons Margin perceptrons SVM	Statistical Learning. Ch 4.	HW2 submission Due Oct 20th Quiz 8 Due Oct 21
Oct 23 slides	Lecture 14: Classification Linear SVM Kernel SVM Multiclass Problem	Statistical Learning. Ch 4.	HW3 out
Oct 25 slides	Lecture 15: Modelling Statistical modelling ML estimation Expectation Maximization Gaussian Mixture Models	Event Detection using GMM	Quiz 9 Due Oct 28
Oct 30 slides	Guest Lecture: Aswin Sankaranarayanan		Midterm Project Report
Nov 1 slides	Guest Lecture: Joseph Keshet		Quiz 10 Due Nov 4
Nov 6 slides	Lecture 16: Classification Bayes classification Naive Bayes Gaussian classifiers Full covariance Gaussians vs diagonal Covariance Shared vs. separate covariances	Statistical Learning. Ch 6.
Nov 8 slides	Lecture 17: Supervised Representations Distinction between supervised and unsupervised models CCA LDA	CCA-based acoustic features for phonetic recognition	Quiz 11 Due Nov 11 HW3 submission Due Nov 12th
Nov 15	Guest Lecture: Roger Dannenberg		HW4 out
Nov 20 slides	Lecture 18: Modelling/Classification Markov Models Hidden Markov Models Training HMMS Classification with HMMs Segmentation with HMMs	Tutorial on HMM
Nov 22	Thanksgiving Day
Nov 27 slides	Lecture 19: Modelling MAP Estimation Linear Gaussian Models	Voice biometrics using linear Gaussian Model	HW4 submission Due Nov 27th
Nov 29 slides	Lecture 20: Modelling/Prediction Linear dynamic models Kalman filters Extended Kalman filters	A tutorial introduction to Kalman filtering Nonlinear filters: beyond the Kalman filter	Quiz 12 Due Dec 2nd Final Project Report Due Dec 6th
Dic 4	Poster Session

Machine Learning for Signal Processing

11-755 / 18-797 Fall 2018

Prof. Bhiksha Raj