Handouts for 15-681, Machine Learning, Fall 1995, Tom Mitchell

Readings and Handouts: Draft chapters of Machine Learning , Tom Mitchell, McGraw Hill, 1996. (handed out during lecture)

• Chapter 1, Introduction (8/29/95)
• Chapter 2, Concept Learning (9/5/95)
• Chapter 3, Computational Learning Theory (9/26/95)
• Chapter 4, Decision Tree Learning (9/12/95)
• Chapter 5, Neural Network Learning (10/10/95)
• Chapter 6, Bayesian Approaches (10/31/95)
• Chapter 7, Genetic Algorithms
• Chapter 8, Inductive Logic Programming
• Chapter 9, Explanation Based Learning (11/16/95)
• Chapter 10, Combining Inductive and Analytical Learning (11/21/95)
• Chapter 11, Reinforcement Learning (12/5/95)

General course handouts:

• Administrivia (8/29/95)

Assignments:

• (Concept learning) Assignment 1. Due September 12. Do exercises 2.1 and 2.2 from chapter 2. Solutions.
• (Programming: Write a decision tree learner) Assignment 2: Due October 3.
• (Decision Tree learning and PAC learning) Assignment 3: Due October 10. Do exercise 1 from chapter 3, and exercise 3 from Chapter 4 See the CORRECTION to this assignment . Solutions.
• (Midterm review) Assignment 4: Due October 17. Solutions.
• (Face recognition and neural networks) Assignment 5: Part I due October 26, Part II due November 2. Click here to see some student results.
• Assignment 6: Due November 21. Solutions.
• (MDL, Knowledge guided ANN learning) Assignment 7: Due December 5. Solutions.

Lecture slides (when available):

• Aug 29. Overview, and design of a checkers learner. (chapter 1)
• Sep 5,12. Concept learning and the general-to-specific ordering. (chapter 2)
• Sep 14. Introduction to learning decision trees: ID3/C4.5. (chapter 4)
• Sep 19,21. Decision tree complexities, Occam's razor. (chapter 4)
• Sep 26. Probably approximately correct learning. (chapter 3)
• Sep 28. VC Dimension. (chapter 3)
• Oct 3. Agnostic learning, Mistake bounded learning. (chapter 3)
• Oct 5. Student presentations: decision tree learning experiments
• Oct 10. Perceptrons, Gradient descent. (chapter 5)
• Oct 12. Backpropagation. (chapter 5)
• Oct 17. Representation in Backprop nets, Midterm review. (chapter 5)
• MIDTERM EXAM - October 19 - Open book
• Oct 24. Weight sharing, TDNN's, recurrent nets, alternative objective functions (chapter 5)
• Oct 26. VC dimension of neural networks, Weighted majority and multiplicative updates. See also Avrim Blum's notes on weighted majority.
• Oct 31. Bayesian reasoning, MAP and ML hypotheses, min sq. error and max likelihood (chapter 6)
• Nov 2. Student presentations: neural network face recognition
• Nov 7. Maximum likelihood neural networks for predicting probabilities (chapter 6)
• Nov 9. Bayes optimal classifier, Gibbs algorithm, Minimum Description Length principle, NewsWeeder (chapter 6)
• Nov 14. Explanation-based learning. (chapter 9)
• Nov 16. Combining inductive and analytical learning (EBNN) (chapter 10)
• Nov 21. Combining inductive and analytical learning (KBANN) (chapter 10)
• Nov 28. Combining inductive and analytical learning (FOIL, FOCL) (chapter 10)
• Nov 30. Evolutionary Computation and Machine Learning (lecture by Matt Glickman)
• Dec 5. Reinforcment Learning (chapter 12)
• Dec 7. Perspectives on Machine Learning
• Dec 10. 1:30-2:30 Optional review for final exam
• December 18 - 5:30-8:30 p.m. FINAL EXAM. OPEN BOOK, OPEN NOTES