Handouts List:

• Administrivia .
• Mitchell, Machine Learning, Chapter 1 draft: Introduction
• Mitchell, Machine Learning, Chapter 2 draft: Concept Learning
• Kearns, Li, Pitt, Valiant: "Recent results on boolean concept learning".
• Littlestone: "Learning quickly...".
• Mitchell, Machine Learning, Chapter 3 draft: Decision Tree Learning
• Murphy, P.M. and Pazzani, M.J. (1994) "Exploring the Decision Forest: An Empirical Investigation of Occam's Razor in Decision Tree Induction", Journal of AI Research, Volume 1, pages 257-275.
• Rivest's lecture notes on Bayesian inference.
• Selection from Minsky and Papert, Perceptrons.
• Rumelhart, Hinton, and Williams "Learning internal representations by error propagation" Chapter 8 of Parallel Distributed Processing, Rumelhart and McClelland.
• Mitchell, et. al., "Experience with a Learning Personal Assistant", CACM, July, 1994.
• Lang, Waibel, and Hinton, "A time-delay neural network architecture for isolated word recognition" Neural Networks, (3), pp. 33-43, 1990.
• Mitchell, Machine Learning, Chapter 8 draft: Genetic Algorithms
• Mitchell, Machine Learning, Chapter 6 draft: Analytical Learning
• Mitchell and Thrun, "Explanation-Based Learning: A Comparison of Symbolic and Neural Network Approaches", Tenth Int. Conf. on Machine Learning, June, 1993.
• Watkins and Dayan, "Q-learning".
• Rivest and Schapire, "Inference of Finite Automata using Homing Sequences", Inf.&Comp. 103, 1993.

Lecture slides (when available):

• 8/30 - Introduction (plus some extra slides we didn't cover).
• 9/1 - Concept Learning and Version Spaces.
• 9/6 and 9/8 - Mistake bound and PAC models of learning.
• 9/13 - Decision Tree Learning I.
• 9/15 - Decision Tree Learning II: overfitting, pruning, etc..
• 9/20 - Occam's Razor I: PAC model view .
• 9/22 - Occam's Razor II: Bayesian view and decision forest experiment.
• 9/27 - Perceptrons and linear functions.
• 9/29 - Neural network backpropagation
• 10/4 - Case study: Decision tree learning for calendar scheduling ( slides, or reading).
• 10/6 - Dealing with large feature sets: Weighted-Majority based methods
• 10/11 - More on Backpropagation
• 10/13 Midterm Open book, open notes
• 10/18 Recurrent neural nets . Notes on probabilistic inequalities .
• 10/20 VC-dimension.
• 10/25 Genetic algorithms .
• 10/27 Analytical learning. .
• 11/1 Combining Inductive and Analytical Learning I.
• 11/3 Combining Inductive and Analytical Learning II.
• 11/8 Student presentations: Learning to recognize faces
• 11/10 Nearest neighbor and case-based reasoning .
• 11/15 Boosting and Learning First Order Hypotheses (FOIL and FOCL) .
• 11/17 Hidden Markov Models .
• 11/22 Reinforcement Learning I .
• 11/29 Mathematics of Q-learning .
• 12/1 Reinforcement Learning II .
• 12/12 FINAL EXAM 5:30 pm

Assignments:

• Assignment 1: Due Sept 8. Problems 1 and 2 from Chapter 2.
• Assignment 2 . Due Sept 15. Note: in problem 1(b), you may use O( ) notation in your answer. Also, in problem 4, the reference should be to 1(a), not 2(a).
• Assignment 3. Decision tree learning of calendar scheduling preferences. Due October 4. See also /afs/cs/project/theo-1/assignment/README. Click here to see code/results/observations that students have made available.
• Assignment 4. Due October 25. ( solutions )
• Assignment 5. Face recognition. Due November 8. What to hand in. The full materials (training images, Backpropagation source code, etc.) are available here.
• Assignment 6. Reinforcement learning. ( solutions )