AI Lunch and Seminar

Jason Hartline - Peer Grading and Mechanism Design

Abstract: The first part of the talk will overview a peer grading system that is under development at Northwestern U. In courses that use the system it has (a) reduced the grading load of course staff by over 75%, (b) expanded and improved the students’ interaction with the course material, (c) and improved turn-around time of feedback on student work (students receive comments on their work after three days, rather than two weeks). As a research platform, this system enables a dialogue between theory and practice for algorithms, machine learning, and mechanism design.

Of particular focus for the talk is a connection between incentivizing students to give accurate peer reviews and all-pay position auctions. All-pay auctions are a common model for competitions based on effort. In an all-pay position auction, bidders compete for positions, where higher positions are more valuable, and all bidders pay their bids (i.e., their effort is spent). The equilibrium outcome of an all-pay auction depends on the relative values of the positions. The second part of the talk will describe recent work on reoptimizing all-pay position auctions from their equilibrium bid data.

The second part of the talk is joint work with Shuchi Chawla and Denis Nekipelov.

Bio: Prof. Hartline received his Ph.D. in 2003 from the University of Washington under the supervision of Anna Karlin. He was a postdoctoral fellow at Carnegie Mellon University under the supervision of Avrim Blum; and subsequently a researcher at Microsoft Research in Silicon Valley. He joined Northwestern University in 2008 where he is an associate professor of computer science. He was on sabbatical at Harvard University in the Economics Department during the 2014 calendar year and visiting Microsoft Research, New England for the Spring of 2015.