10708 Probabilistic Graphical Models

Course Description

Many of the problems in artificial intelligence, statistics, computer systems, computer vision, natural language processing, and computational biology, among many other fields, can be viewed as the search for a coherent global conclusion from local information. The probabilistic graphical models framework provides an unified view for this wide range of problems, enabling efficient inference, decision-making and learning in problems with a very large number of attributes and huge datasets. This graduate-level course will provide you with a strong foundation for both applying graphical models to complex problems and for addressing core research topics in graphical models.

The class will cover three aspects: the core representation, including Bayesian and Markov networks, and dynamic Bayesian networks; probabilistic inference algorithms, both exact and approximate; and learning methods for both the parameters and the structure of graphical models. Students entering the class should have a pre-existing working knowledge of probability, statistics, and algorithms, though the class has been designed to allow students with a strong mathematical background to catch up and fully participate.

It is expected that after taking this class, students will have obtained sufficient working knowledge of multi-variate probablistic modeling and inference for practical applications, should be able to fomulate and solve a wide range of problems in their own domain using graphical models, and can advance into more specialized technical literature by themselves.

In order to take this class, students are required to have successfully completed Machine Learning 10-701/15-781, or an equivalent class.

Textbooks

The required textbook for this class is (note that the material of the class goes beyond this book):

Daphne Koller and Nir Friedman, Probabilistic Graphical Models: Principles and Techniques

We will also be using excerpts from the following work, which you do not need to purchase:

Michael I. Jordan, An Introduction to Probabilistic Graphical Models, in preparation. Copies of certain chapters will be made available.

Optional:

Kevin Murphy, Machine Learning: a Probabilistic Perspective

Grading

The class requirements include brief daily reading summaries, scribe notes for 1 lecture, 5 problem sets, and a project. This is a PhD level course, and by the end of this class you should have a good understanding of the basic methodologies in probabilistic graphical models, and be able to use them to solve real problems of modest complexity. The grading breakdown is as follows:

Reading Summaries (10%)
Scribe Duties (10%)
Homework Assignments (40%)
Final Project (40%)

Note that this class does not have any exams.

Reading Summaries

The required readings for this class are compulsory. At the beginning of each lecture, you are to submit a half-page summary of the readings for that lecture. These reading summaries are a requirement for this class, and a hard copy must be turned in on time (i.e. before the start of class) by you in order to receive credit. However, you have a total of 5 grace days on which you can submit the summaries after the start of class or via email to 10708-instructor@cs.cmu.edu (e.g. if you need to absent from class for whatever reason) with no penalty.

Your summary should be written at a high level, and should focus on the main point of the readings (i.e. avoid complicated math). As long as your summary is reasonable, you will be given full credit.

Scribe Duties

Each student is required to scribe for a small number of lectures (most likely just 1). Most lectures will have at least 2 students acting as scribes, and they should work as a team. During your assigned lectures, you are to take detailed notes in collaboration with your fellow scribes. After the lecture, the scribe team is to convert their notes into LaTeX format using this template. These notes should be at least 6-8 pages long, and must be submitted electronically to the instructor email list within 1 week after the lecture. We only require one set of notes from the scribe team. The instructors will then audit your notes, and post them to the class page for everyone's benefit.

As long as your scribe notes are of sufficient standard, you will be awarded full credit for scribe duties. If your notes have errors or are otherwise not up to standard, we will inform you and give you a chance to correct them. You will receive zero credit if you fail to submit your notes.

Homeworks

There will be 4 homework assignments over the course of the semester. These homeworks may contain material that has been covered by published papers and webpages. Since this is a graduate class, we expect students to solve the problems themselves rather than search for answers online.

Homeworks will be done individually: each student must hand in their own answers. It is acceptable, however, for students to collaborate in figuring out answers and helping each other solve the problems. We will be assuming that, as participants in a graduate course, you will be taking the responsibility to make sure you personally understand the solution to any work arising from such collaboration. You also must indicate on each homework with whom you collaborated.

We strongly recommend that you typeset your homework using appropriate software such as LaTeX. If you are writing please make sure your homework is cleanly written up and legible. The TAs will not invest undue effort to decrypt bad handwriting.

Final Project

The class project will be carried out in groups of 2 or 3 people, and has four main parts: a proposal, a midway report, a final report, and a poster/oral presentation. The project is an integral part of this class, and is designed to be as similar as possible to researching and writing a conference-style paper. Please see the project page for more information about the final project.

Collaboration Policy

Homeworks should be done individually: each student must hand in their own answers. It is acceptable, however, for students to collaborate when figuring out answers and to help each other solve the problems. We will be assuming that, as participants in a graduate course, you will be taking the responsibility to make sure you personally understand the solution to any work arising from such collaboration. You also must indicate on each homework with whom you collaborated.

Late Policy

You will be allowed 2 total homework late days without penalty for the entire semester. You may be late by 1 day on two different homeworks or late by 2 days on one homework. Once those days are used, you will be penalized according to the following policy:

Homework is worth full credit at the due time on the due date.
It is worth half credit for the next 48 hours.
It is worth zero credit after that.

You must turn in at least 3 of the 4 homeworks, even if for zero credit, in order to pass the course. Please upload your late submissions to Gradescope.

Regrade Policy

If you feel that we have made an error in grading your homework, please submit a regrading request on Gradescope and we will consider your request. Please note that regrading of a homework may cause your grade to go either up or down.

Auditing

To satisfy the auditing requirement, you must do *one* of the following:

Submit three homeworks, and receive at least 75% of the points on each one.
Do a class project, which must address a topic related to machine learning and must be something that you have started while taking this class (i.e. it can't be something you did last semester). You will need to submit a project proposal with everyone else, and present a poster with everyone. However, you don't need to submit a milestone or final paper. You must get at least 80% on the poster presentation part of the project.

If you plan to audit the class, please send the instructors an email saying that you will be auditing and telling us which requirement you plan to fulfill.

Note to People Outside CMU

Please feel free to reuse any of these course materials that you find of use in your own courses. We ask that you retain any copyright notices, and include a written notice indicating the source of any materials you use.