Syllabus and (tentative) Course Schedule

 

Date	Lecture	Topics	Readings and useful links	Anouncements
Block 1:	Supervised Learning
Mon 9/8	Lecture 1: Regression: Linear and Logistic Eric Xing: slides, annotated slides		No Reading assignment due Bishop, PRML: Ch 4, Ch 5 Mitchell: Ch 4 Chapter (Draft) from Mitchell On Discriminative and Generative Classifiers Ng, Jordan
Wed 9/10	Lecture 2: Linear Regression and Lasso Eric Xing: slides annotated slides		Tutorial on Regression by Andrew Moore Bishop, PRML: Ch 3 Mitchell: Ch 8.3 Regression Shrinkage and Selection via the Lasso by Rob Tibshirani Model Selection and Estimation in Regression with Grouped Variables by Yuan, Lin Large Scale Online Learning by Bottou, Le Cun Feature Selection for High-Dimensional Genomic Microarray Data by Xing, Jordan, Karp On Model Selection Consistency of Lasso by Peng Zhao, Bin Yu
Mon 9/15	Lecture 3: Structured sparsity with application in Computational Genomics Eric Xing: slides		Statistical Estimation of Correlated Genome Associations to a Quantitative Trait Network by S. Kim and E. P. Xing Tree-Guided Group Lasso for Multi-Response Regression with Structured Sparsity, with applications to eQTL Mapping by S. Kim and E. P. Xing Smoothing Proximal Gradient Method for General Structured Sparse Regression by X. Chen, Q. Lin, S. Kim, J. Carbonell and E. P. Xing
Wed 9/17	Lecture 4: Perceptron, Deep Neural Networks Barnabas Poczos: MultiLayerPerceptron DeepArchitectures		Learning Deep Architectures for AI by Yoshua Bengio ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky et. al. Multilayer Feedforward Networks are Universal Approximators by Kur Hornik A Logical Calculus of the ideas immanent in Nervous Activity by Warren S. McCulloch, Walter Pitts The Perceptron: A probabilistic model for information storage and organization in the brain by F. Rosenblatt Optional: Some slides by Eric on Learning DNNs.	Homework 1 out
Block 2:	Kernel Machines
Mon 9/22	Lecture 5: SVMs and Duality Barnabas Poczos: SupportVectorMachines Duality		Required Burges, Christopher J. C.; A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery, 1998 Optional Learning with Kernels Support Vector Machines, Regularization, Optimization, and Beyond. Bernhard Scholkopf and Alexander J. Smola, MIT Press, 2002. Advances in Kernel Methods - Support Vector Learning Edited by Chris Burges, Bernhard Scholkopf and Alexander J. Smola, MIT Press, 1998. Support Vector Machines and other kernel-based learning methods John Shawe-Taylor and Nello Cristianini - Cambridge University Press, 2000
Wed 9/24	Lecture 6: The Kernel Trick & RKHS Eric Xing: slides annotated slides		Required Learning with Kernels, Scholkof & Smola, Ch 2
Mon 9/29	Lecture 7: Reproducing Kernel Hilbert Space Eric Xing
Wed 10/1	Lecture 8: Learning with Kernels Barnabas Poczos			Homework 1 due Homework 2 out
Block 3:	Unsupervised Learning, Density estimation, Graphical Models
Mon 10/6	Lecture 9: Clustering, mixture models, the EM algorithm Barnabas Poczos: slides		Required Max Welling's notes on Clustering and EM.
Wed 10/8	Lecture 10: Clustering, mixture models, the EM algorithm Barnabas Poczos: Slides same as above
Mon 10/13	Lecture 11: Structured Models: Hidden Markov Models vs. Conditional Random Fields Eric Xing: slides		Required Chap. 12 from Michael Jordan's book Chap 12 Shallow Parsing with Conditional Random Fields Optional Rabiner, Lawrence R. (1989). A Tutorial on Hidden Markov Model and selected Applications in Speech Recognition Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Wed 10/15	Lecture 12: Structured Models: Hidden Markov Models vs. Conditional Random Fields, Graphical Models Eric Xing: CRF GraphicalModels		Required Michael Jordan's Introduction to Graphical Models	Homework 2 due
Mon 10/20	Lecture 13: Graphical Models, Markov Chain Monte Carlo and Topic Models Eric Xing: slides		Required Mackay Textbook, Ch. 29.1 - 29.5 Probabilistic Topic Models David Blie Optional Eric's talk in ACL
Wed 10/22	Lecture 14: Markov Chain Monte Carlo Eric Xing: slides (cont'd)
Mon 10/27	Mid Term
Block 4:	Latent Space Analysis, Eigen space analysis
Wed 10/29	Lecture 15: Principal Component Analysis Barnabas Poczos: slides		Required A Tutorial on Principal Component Analysis, Jon Shlens. pdf Optional Kernel Principal Components Analyis, Max Welling. pdf	Homework 3 out
Mon 11/3	Lecture 16: Independent Component Analysis Barnabas Poczos: slides		Required Independent Component Analysis, Aapo Hyvarinen, Erkki Oja. pdf
Wed 11/5	Lecture 17: Independent Component Analysis Barnabas Poczos: slides continued from previous lecture
Block 5:	Bayesian Nonparametrics
Mon 11/10	Lecture 18: Gaussian Processes Barnabas Poczos: slides		Required Gaussian Processes for Machine Learning, Rasmussen, Williams. Chapter 2
Wed 11/12	Non-parametric Bayesian Models Eric Xing: slides		Required Bayesian Haplotype Inference via the Dirichlet Process by Xing et al., ICML 2004 pdf Hierarchical Dirichlet Processes by Teh et al., JASA 2006 pdf Optional Markov Chain Sampling Methods for Dirichlet Process Mixture Models by Radford Neal, 2000 pdf A Constructive Definition of Dirichlet Priors by Sethuraman, 1994 pdf Bayesian Density Estimation and Inference using Mixture models by Escobar et al., 1995 pdf	Homework 3 due Homework 4 out (Fri 11/14)
Mon 11/17	Spectral clustering Eric Xing: slides		Required Normalized Cuts and Image Segmentation by Shi and Malik, 2000 pdf On Spectral Clustering: Analysis and an Algorithm by Ng, Jordan and Weiss, 2001 pdf
Block 6:	Computational Learning theory
Wed 11/19	Risk Minimization Barnabas Poczos: slides		Required Sections 1-3 from Introduction to Statistical Learning Theory by Bousquet et al. pdf	Homework 4 due (Fri 11/21)
Mon 11/24	VC theory Barnabas Poczos slides
Wed 11/26	Thanks Giving Holiday
Mon 12/1	Manifold Learning Barnabas Poczos slides		Required: Spectral Methods for Dimensionality Reduction by Saul et. al pdf Optional: A Global Geometric Framework for Nonlinear Dimensionality Reduction by Tenenbaum et. al pdf Nonlinear dimensionality reduction by locally linear embedding by Roweis et. al pdf Laplacian eigenmaps for dimensionality reduction and data representation by Belkin et. al pdf
Block 7:	Ensemble methods
Wed 12/3	Boosting, random forests
Additional:	If we have more time
	Online Learning
	Local linear embedding, and manifold learning

 

© 2012 Eric Xing @ School of Computer Science, Carnegie Mellon University
[validate xhtml]