10-709 Fall 2017: Fundamentals of Learning from the Crowd



Time: Tuesday and Thursday 1.30pm to 2.50pm
Location: GHC 4303
Units: 12

Instructor: Nihar Shah
Nihar's office hours: 3-4pm every Tuesday in GHC 8211
nihars at cs dot cmu dot edu

TA: Ritesh Noothigattu
Ritesh's office hours: 3-4pm every Thursday outside GHC 8013
riteshn at cmu dot edu

Description: Crowdsourcing is a burgeoning area that is popular in academic research, industrial applications, and also in societal causes. In this course, we will cover the foundational theoretical principles behind crowdsourcing and learning from the crowd. We will study this field via the lens of game theory (how to incentivize people to provide better data), learning theory (how to make sense of this data), and social choice theory (how to be fair). We will also touch upon literature in psychology and economics that studies the behavior of people. Along the way, we will discuss several fascinating paradoxes and conduct some live experiments in the class. Lectures will be taught on the board.

Evaluation: Homeworks, final project, class participation.
Prerequisites: Basic probability (e.g., the student should be comfortable with conditional expectations, the Gaussian distribution, union bound), basic linear algebra (e.g., singular value decomposition) and basic programming. All other required background in game thoery, learning theory, etc. will be taught in the lectures.
Class material: Will be posted on Autolab.

Tentative schedule (subject to change):
Sept 5What is this course about? What is crowdsourcing?
Sept 7How to win $40,000?
Sept 12How to properly make strict rules?
Sept 14How do these rules help in machine learning?
Sept 19How do casinos help in crowdsourcing?
Sept 21Doubling down on casinos helping in crowdsourcing.
Sept 26What was "A beautiful mind" all about?
Sept 28When Thomas met John.
Oct 3How to administer a virtual truth serum?
Oct 5Some more serum for dessert please.
Oct 10What are concentration inequalities? And no, its not related to Yoga.
Oct 12What are good models for ranking?
Oct 17How to rank in a simple, robust and optimal manner?
Oct 19How to become more active?
Oct 24How to grade your peers? (Guest lecture by Ritesh Noothigattu)
Oct 26Who started the gossip?
Oct 31I started the gossip. And you can't catch me. (Guest lecture by Prof. Giulia Fanti)
Nov 2The labels are too crowded
Nov 7Whats up with these parameters?
Nov 9The world is a paradox.
Nov 14We are not who we really think we are. Then who are we?
Nov 16How can I be fair to everyone?
Nov 21You cannot be fair to everyone.
Nov 28Why won't you vote for your favorite candidate?
Nov 30Finishing up. Do you have any questions?
Dec 5Project presentations I
Dec 7Project presentations II


Timeline for homeworks and project:
Project:
Sept 20, 1pmGroup names
Oct 17, 1pmProposal
Dec 5, 1pmFinal report
Last two lecturesPresentation
Major homeworks:
Sept 20, 1pmMajor homework 1 release
Oct 8, 1pmMajor homework 1 submission
Nov 3, 1pmMajor homework 2 release
Nov 21, 1pmMajor homework 2 submission


Final Projects: