Course Description

The rapid improvement of sensory techniques and processor speed, and the availability of inexpensive massive digital storage, have led to a growing demand for systems that can automatically comprehend and mine massive and complex data from diverse sources. Machine Learning is becoming the primary mechanism by which information is extracted from Big Data, and a primary pillar that Artificial Intelligence is built upon.

This course is designed for Ph.D. students whose primary field of study is machine learning, or who intend to make machine learning methodological research a main focus of their thesis. It will give students a thorough grounding in the algorithms, mathematics, theories, and insights needed to do in-depth research and applications in machine learning. The topics of this course will in part parallel those covered in the general graduate machine learning course (10-701), but with a greater emphasis on depth in theory and algorithms. The course will also include additional advanced topics such as RKHS and representer theory, Bayesian nonparametrics, additional material on graphical models, manifolds and spectral graph theory, reinforcement learning and online learning, etc.

Students entering the class are expected to have a pre-existing strong working knowledge of algorithms, linear algebra, probability, and statistics. If you are interested in this topic, but do not have the required background or are not planning to work on a PhD thesis with machine learning as the main focus, you might consider the general graduate Machine Learning course (10-701) or the Masters-level Machine Learning course (10-601).

Textbook

There is no required textbook for this course. Most content will be covered in class and additional reading material will be made available as appropriate. We recommend David Mackay's Information Theory, Inference, and Learning Algorithms (available free online here) as optional reading.

Grading

The requirements of this course consist of participating in lectures, 4 problem sets, a project and a mid term exam.
The breakdown is as follows:

• Homeworks: 40%
• Midterm: 20%
• Final project: 40%

Auditing

Since we already have a long waitlist, priority will be given to students who have already registered for the course. Auditing requirements will be made available shortly. If you plan to audit the course, please send the instructors an email saying that you will be auditing the class as soon as possible.

Prerequisites

This course has no official prerequisites. However, we will assume a fair amount of expertise in several topics on Linear Algebra, Calculus, Probability and Statistics.

We have prepared this document with a list of topics we will be assuming familiarity with when we conduct this course. We already have a fairly long waitlist. If you have already been registered, we recommend that you do your own self assessment to see if you have the background to take this course. If you feel that this will be challenging, please consider taking the introductory Machine Learning Class (10-701) so that we can allow people on the waitlist in.

Homework resources and collaboration policy

Homeworks and exams may contain material that has been covered by papers and webpages. Since this is a graduate class, we expect students to want to learn and not google for answers.

Homeworks will be done individually: each student must hand in their own answers. It is acceptable, however, for students to collaborate in figuring out answers and helping each other solve the problems. We will be assuming that, as participants in a graduate course, you will be taking the responsibility to make sure you personally understand the solution to any work arising from such collaboration. You also must indicate on each homework with whom you collaborated.

The Course Project should be done in groups of 2-3 students.

Late homework policy

• You will be allowed 2 total late days without penalty for the entire semester.  You may be late by 1 day on two different homeworks or late by 2 days on one homework. Once those days are used, you will be penalized according to the following policy:
  • Homework is worth full credit at the beginning of class on the due date.
  • It is worth half credit for the next 48 hours.
  • It is worth zero credit after that.
• You must turn in at least n-1 of the n homeworks, even if for zero credit, in order to pass the course.
• Turn in all late homework assignments to Mallory Deptola. For those who have to submit homework at weekends, email assignment to 10715-instructors@cs

Homework regrades policy

If you feel that we have made an error in grading your homework, please turn in your homework with a written explanation to  Mallory Deptola and we will consider your request. Please note that regrading of a homework may cause your grade to go up or down.

Note to people outside CMU

Please feel free to reuse any of these course materials that you find of use in your own courses.  We ask that you retain any copyright notices, and include written notice indicating the source of any materials you use.