Advanced Introduction to Machine Learning

10-715, Fall 2018

Carnegie Mellon University

Maria-Florina Balcan

Previous material

This is a tentative schedule and is subject to change.

Date	Lecture	Topics	Readings and useful links	Handouts
Aug 27 (M)	Course Overview	Intro, admin Decision Tree Learning	Mitchell Chapters 1, 2, 6.1-6.3 Murphy Chapter 2 Bishop Chapter 1, 2	Slides
Aug 29 (W)	Learning Linear Separators	Learning Linear Separators The Perceptron Algorithm Geometric Margins	Chapter 9.1.1. and 9.1.2, Shalev-Shwartz and Ben-David Mitchell Chapters 4.1.2 and 4.4.1 Bishop Chapter 4.1.7	Slides
Sept 3 (M)	No Class: Labour Day
Sept 5 (W)	Probability and Estimation, Naive Bayes	Estimating Probabilities from Data: MLE, MAP Naive Bayes, Conditional Independence	Mitchell: Estimating Probabilities Murphy, chapter 3,4 Mitchell: Naive Bayes	Slides Slides
Sept 10 (M)	Generative and Discriminative Classifiers, Logistic Regression, Naive Bayes	Naive Bayes, Text Classification and Bag of Words Representation Logistic Regression: Maximizing Conditional Likelihood	Mitchell: Naive Bayes, Logistic Regression Chapter 4.4 in Tibshirani Chapter 14 of Shalev-Shwartz and Ben-David	Slides Slides
Sept 12 (W)	Kernels	Kernels Kernelizing Algorithms Kernelizing Perceptron	Bishop 6.1 and 6.2 Nina's old notes Lanckriet et al.	Slides
Sept 17 (M)	Generalization and Overfitting	Sample Complexity	Chapter 7, Mitchell Chapters 2,3,4 Shalev-Shwartz and Ben-David	Slides
Sept 19 (W)	Generalization and Overfitting	Sample Complexity VC Dimension Based Bounds	Chapter 7, Mitchell Chapters 2,3,4 Shalev-Shwartz and Ben-David	Slides
Sept 24 (M)	Generalization and Overfitting	Sample Complexity Rademacher Based Bounds Model Selection	Chapter 26 of Shalev-Shwartz and Ben-David Nina's old notes	Slides
Sept 26 (W)	Support Vector Machines	Primal and Dual Forms Kernalizing SVM	Chapter 15 of Shalev-Shwartz and Ben-David Notes on SVM by Andrew Ng	Slides
Oct 1 (M)	Boosting	Weak learning, Strong learning, Adaboost	Shapire's NIPS Tutorial Shapire's Survey on Boosting	Slides
Oct 3 (W)	Boosting, Model Selection	Margin based bounds for Boosting k-fold cross validation Structural risk minimization	Chapters 10 and 11 of Shalev-Shwartz and Ben-David	Slides Slides Slides
Oct 8 (M)	Midterm
Oct 10 (W)	Linear Regression	Linear Regression Minimizing squared error and maximizing data likelihood	Murphy: Chapter 7.1-7.3	Slides
Oct 15 (M)	Neural Networks	Neural Networks Backpropagation	Mitchell: Chapter 4 Goodfellow: Chapter 9	Slides
Oct 17 (W)	Deep Networks	Convolution Convolutional Neural Networks	Mitchell: Chapter 4 Goodfellow: Chapter 9	Slides Slides
Oct 22 (M)	Active Learning	Active Learning Common heuristics, Sampling bias Safe Disagreement Based Active Learning Schemes	Two Faces of Active Learning by Sanjoy Dasgupta Active Learning - Modern Learning Theory by Balcan and Urner, 2015 Sample and Computationally Efficient Active Learning	Slides
Oct 24 (W)	Semi-Supervised Learning	Semi-Supervised Learning Transductive SVM Co-training	Semi-Supervised Learning by Jerry Zhu Combining Labeled and Unlabeled Data with Co-Training A Discriminative Model for Semi-Supervised Learning A PAC Framework for Semi-Supervised Learning	Slides
Oct 29 (M)	Graphical Models (Guest lecture by Matt Gormley)	Bayesian Networks Topic Models	TBD	Slides
Oct 31 (W)	Graphical Models (Guest lecture by Matt Gormley)	Hidden Markov Models Conditional Random Fields	TBD	Slides
Nov 5 (M)	Unsupervised Learning	Partitional Clustering Hierarchical Clustering	Murphy 25.5 Hastie, Tibshirani and Friedman, Chapter 14.3 Center Based Clustering: A Foundational Perspective	Slides
Nov 7 (W)	Dimensionality Reduction	Principal Component Analysis Kernel Principal Component Analysis	Bishop 12.1, 12.3 Chapter 23 in Shalev-Shwartz and Ben-David book	Slides
Nov 12 (M)	Online Learning	Online Learning	Avrim Blum survey Chapter 21 of Shalev-Shwartz and Ben-David	Slides
Nov 14 (W)	Deep Unsupervised Learning (Guest lecture by Russ)	Deep Unsupervised Learning	TBD	Slides
Nov 19 (M)	Reinforcement Learning	Markov Decision Processes Value Iteration Q-Learning	Mitchell, Chapter 13 Kaelbling, et al., Reinforcement Learning: A Survey	Slides
Nov 21 (W)	No Class: Thanksgiving
Nov 26 (M)	Project Presentations
Nov 28 (W)	Project Presentations
Dec 3 (M)	Final
Dec 5 (W)	Differential Privacy	Differential Privacy	TBD	Slides