Machine Learning is concerned with computer programs that automatically improve their performance through experience. This course covers the theory and practice of machine learning from a variety of perspectives. We cover topics such as learning decision trees, neural network learning, statistical learning methods, genetic algorithms, Bayesian learning methods, explanation-based learning, and reinforcement learning. The course covers theoretical concepts such as inductive bias, the PAC and Mistake-bound learning frameworks, minimum description length principle, and Occam's Razor. Programming assignments include hands-on experiments with various learning algorithms. Typical assignments include neural network learning for face recognition, and decision tree learning from databases of credit records.
 

Time and Place: Tues & Thurs 12:00-1:20, Porter Hall 125C

Instructor:

Teaching Assistants:

Course Secretary:

Textbook: Machine Learning, Tom Mitchell, McGraw Hill, 1997.

This course is a combination upper-level undergraduate and introductory graduate course.
Ph.D. students in CS may obtain one core credit unit by arranging an extra course project.

Grading: Will be based on homeworks (35%), midterm (30%), and final (35%).

Policy on late homework:

• Homework is worth full credit at the beginning of class on the due date,
• It is worth half credit for the next 48 hours,
• It is worth zero credit after that.
• You must turn in all but two assignments, even if for zero credit
• Free exemption: We will ignore your lowest homework grade for the semester.

Homework assignments (postscript)

• Assignment 1 . Due Sept 11, 1997. Concept learning, sample complexity.
• Assignment 2 . Due Sept 25. Decision tree learning, credit analysis dataset. 
• Assignment 3. Due Oct 16. Neural Network learning, face recognition dataset. 
• Assignment 4. Due Oct 28. Confidence intervals and Bayesian reasoning. 
• Assignment 5. Due Nov 13. Genetic algorithms, Inductive logic programming. 
• Assignment 6. Due Dec 2. Naive Bayes for text classification. 

Lecture Notes (postscript)

• Aug 26, 1997. Overview and design of a checkers learner. (Chapter 1)
• Aug 28. No lecture today.
• Sept 2 and 4. Learning concepts, Version Spaces, inductive bias. (Chapter 2)
• Sept 9. Sample complexity, PAC framework. (Chapt 7, sections 7.2 and 7.3 up to (not including) 7.3.1.)
• Sept 11. PAC learning, VC dimension (remainder of 7.3 except for 7.3.3.2, read 7.4 through 7.4.3, read 7.5 through 7.5.2)
• Sept 16. Decision trees (Chapter 3 through 3.6)
• Sept 18. Decision trees II ( Chapter 3.7 through 3.8)
• Sept 23. Neural networks (Chapter 4)
• Sept 25. Neural networks II (Chapter 4)
• Sept 30. Neural nets III. (Chaper 4)
• Oct 2. Review for Midterm
• Oct 7. Midterm exam. in class, open notes, open book.
• Oct 9. Estimation and confidence intervals (Chapter 5). Guest lecture: Prof. Meyer
• Oct 14. Estimation and confidence intervals II (Chapter 5), and relation to PAC model
• Oct 16. Bayesian learning methods (Chapter 6).
• Oct 21. Bayesian learning methods II (Chapter 6).
• Oct 23. Bayesian learning methods III (and text classification) (Chapter 6).
• Oct 28. Instance-based learning (Chapter 8). Guest lecture: Frank Dellaert
• Oct 30. Genetic algorithms, Genetic programming (Chapter 9).
• Nov 4. Genetic algorithms II (Chapter 9).
• Nov 6. Learning Sets of Rules, ILP (Chapter 10).
• Nov 11. Explanation based learning (Chapter 11 thru 11.3)
• Nov 13. Combining Inductive and Analytical Learning (Chapter 12 thru 12.3.)
• Nov 18. Bayesian Belief Networks, EM (Chapter 6.11, 6.12)
• Nov 20. Hidden Markov Models. Guest lecture: Prof. Rosenfeld
• Nov 25. Reinforcement learning (Chapter 13)
• Dec 2. Reinforcement learning II (Chapter 13), Wrapup: Summary and Perspectives
• Dec 4. No lecture. Optional review session for final.
• Dec 9. Final exam. 5:30-8:30 pm, Wean 7500, open notes, open book.

Note to people outside CMU

 See also the Fall 1996 version of this course , co-taught with Prof. Avrim Blum.