Scott Davies
(Wean 5103, office hours: Mon 2-3)
Course textbook:
Machine Learning, Tom Mitchell, McGraw Hill
Copies of the textbook can be picked up in Jean Harpley's office: Wean
Hall 5313.
GENERAL DESCRIPTION
Machine Learning is concerned with computer programs that automatically
improve their performance through experience. Machine Learning methods have
been applied to problems such as learning to drive an autonomous vehicle,
learning to recognize human speech, learning to detect credit card fraud, and
learning strategies for game playing. This course covers the primary
approaches to machine learning from a variety of fields, including inductive
inference of decision trees, neural network learning, statistical learning
methods, genetic algorithms, bayesian methods, explanation-based learning, and
reinforcement learning. The course will also cover theoretical concepts such
as inductive bias, the PAC and Mistake-Bound learning framework,
Occam's Razor, uniform convergence, models of noise, and Fourier
analysis. Programming assignments include experimenting
with various learning problems and algorithms. This course is a combination
upper-level undergraduate and introductory graduate course. CS Ph.D. students
can obtain one core credit unit by arrangement with the instructor.
Here is a Course syllabus.
Handouts
Assignment Updates
You should check the Assignment
Update page periodically for important announcements regarding
current assignments.
Lecture notes (postscript)
- Aug 27.
Overview and design of a checkers learner. (Read Chapter 1)
- Aug 29.
Concept learning, version spaces, inductive bias. (Read Chapter 2)
- Sept 3. Theoretical frameworks:
consistency and PAC models. (Read Chapter 7.1-7.3)
- Sept 5. PAC model contd, Decision lists,
Occam's razor.
- Sept 10.
Decision tree learning (Read Chapter 3)
- Sept 12.
Subtleties of decision tree learning. (Read Chapter 3)
- Sept 17.
Evaluating hypotheses. (Read Chapter 5.1-5.4)
- Sept 19.
More on evaluating hypotheses (Rest of Chapter 5), online learning, weighted majority algorithm.
- Sept 24. Weighted majority algorithm and
applications. (Read Chapter 7.5)
- Sept 26. More on on-line learning,
Winnow algorithm.
- Oct 1. Neural networks (Chapter 4.1-4.5.1).
- Oct 3. Neural networks (Chapter 4.5.2-4.6.4 (skip 4.5.3)).
- Oct 8. MIDTERM EXAM (in class, open book) Grading info. Solutions.
- Oct 10. Neural nets, C[m], "effective
degrees of freedom", VC-dimension. (Chapter 7.4)
- Oct 15.
Bayesian learning (Chapter 6)
- Oct 17.
More Bayes, Max Likelihood and sum of squared error, Minimum Description Length Principle (Chapter 6)
- Oct 22. Finish Bayes: Bayes optimal,
relation to WM, using Naive Bayes to classify text (Chapter 6).
- Oct 24. Boosting: Theory and applications.
- Oct 29. Class presentations: neural nets and face recognition (assignmt 5)
- Oct 31. Nearest neighbor (Chapter 8)
- Nov 5.
Genetic algorithms, Genetic programming (Chapter 9)
- Nov 7.
Schema Theorem, Sequential Covering Algorithms (Chapt 9, 10)
- Nov 12.
Inductive Logic Programming, FOIL, CIGOL (Chapter 10).
- Nov 14. Active learning, learning
finite state devices.
- Nov 19.
Explanation Based Learning (Chapter 11).
- Nov 21.
Combining inductive and analytical learning; KBANN (Chapter 12).
- Nov 26.
Combining inductive and analytical learning; EBNN, FOCL (Chapter
12), and The EM (Estimation/Maximization)
Algorithm.
- Dec 3. Hidden Markov Models.
- Dec 9.
FINAL EXAM
8:30-11:30, MM 103
Note to people outside CMU
Feel free to use the slides and materials available online here.
Please email
Tom.Mitchell@cmu.edu or
avrim@cs.cmu.edu with any corrections or improvements.
See also Fall 1995 version of this course, including midterm and final exam.