Publications

(Email me at ebrun AT cs dot cmu dot edu for any papers listed without links)

2017

• Coming soon: 2 AAAI papers, 1 IAAI paper, 1 Learning at Scale paper

2016

• Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning [link, reviews]
  P.Thomas and E.Brunskill
  to appear in International Conference on Machine Learning (ICML) 2016
• Energetic Natural Gradient Descent [link, reviews]
  P.Thomas, C.Dann, B. Castro da Silva and E.Brunskill
  to appear in International Conference on Machine Learning (ICML) 2016
• Efficient Bayesian Clustering for Reinforcement Learning [link]
  Travis Mandel, Yun-En Liu, Emma Brunskill and Zoran Popovic
  to appear in International Joint Conference on AI (IJCAI) 2016
• Latent Contextual Bandits and their Application to Personalized Recommendations for New Users [pdf]
  Li Zhou and Emma Brunskill
  to appear in International Joint Conference on AI (IJCAI) 2016
• PAC Continuous State Online Multitask Reinforcement Learning with Identification [link, supplemental]
  Yao Liu, Zhaohan Guo, and Emma Brunskill
  AAMAS 2016
• A PAC RL Algorithm for Episodic POMDPs [link]
  Zhaohan Guo, Shayan Doroudi and Emma Brunskill
  AISTATS 2016
• Offline Evaluation of Online Reinforcement Learning Algorithms [link]
  Travis Mandel, Yun-En Liu, Emma Brunskill and Zoran Popovic
  AAAI 2016
• Sequence Matters, But How Exactly? A Methodology for Evaluating Activity Sequences from Data
  Shayan Doroudi, Kenneth Holstein, Vincent Aleven and Emma Brunskill
  to appear in Educational Data Mining (EDM) 2016
• Questimator: Generating Knowledge Assessments for Arbitrary Topics
  Qi Guo, Chinmay Kulkarni, Aniket Kittur, Jeffrey Bigham and Emma Brunskill
  to appear in International Joint Conference on AI (IJCAI) 2016
• Interface Design Optimization as a Multi-Armed Bandit Problem
  James Lomas, Jodi Forlizzi, Nikhil Poonwala, Nirmal Patel, Sharan Shodhan, Kishan Patel, Ken Koedinger, and Emma Brunskill
  to appear in Computer Human Interaction (CHI) 2016.
• Towards a Learning Science for Complex Crowdsourcing Tasks
  Shayan Doroudi, Ece Kamar, Emma Brunskill and Eric Horvitz
  to appear in Computer Human Interaction (CHI) 2016.
• Automatically Learning to Teach to the Learning Objective [pdf]
  Rika Antonova, Joe Runde, Dexter Lee, and Emma Brunskill
  work in progress to appear in Learning at Scale 2016
• Faster Teaching via POMDP Planning
  Anna Rafferty, Emma Brunskill, Tom Griffiths and Pat Shafto
  accepted Cognitive Science

2015

• Sample Complexity of Episodic Fixed-Horizon Reinforcement Learning [arxiv pdf]
  Christoph Dann and Emma Brunskill
  NIPS 2015
• Concurrent PAC RL [pdf]
  Zhaohan (Daniel) Guo and Emma Brunskill.
  AAAI 2015
• The Queue Method: Handling Delay, Heuristics, Prior Data, and Evaluation in Bandits[link]
  Travis Mandel, Yun-En Liu, Emma Brunskill, Zoran Popovic.
  AAAI 2015
• From Predictive Models to Instructional Policies [pdf soon]
  Joseph Rollinson and Emma Brunskill
  Educational Data Mining (EDM) 2015.
• Towards Understanding How to Leverage Sense-making, Induction/Refinement and Fluency to Improve Robust Learning [pdf soon]
  Shayan Doroudi, Kenneth Holstein, Vincent Aleven and Emma Brunskill.
  Educational Data Mining (EDM) 2015.
• Learning the Features Used to Decide How to Teach [link]
  Min Hyung Lee, Joe Runde, Warfa Jabril, Zhouying Wang, and Emma Brunskill.
  Learning at Scale 2015. Work in Progess paper.

2014

• PAC-inspired Option Discovery in Lifelong Reinforcement Learning [pdf]
  Note:Lazaric and Fruit pointed out that PAC-SMDP and PAC-MDP are not always directly comparable, hence our paper's analysis of the benefit of options over primitive actions does not always hold.
  E.Brunskill and L.Li
  ICML 2014
• Resource-Efficient Stochastic Optimization of a Locally Smooth Function under Correlated Bandit Feedback [pdf]
  M.Azar, A.Lazaric and E.Brunskill
  ICML 2014
• Trading Off Scientific Knowledge and User Learning with Multi-Armed Bandits [pdf]
  Yun-En Liu, Travis Mandel, Emma Brunskill, and Zoran Popovic.
  Educational Data Mining (EDM) 2014.
• Towards Automatic Experimentation of Educational Knowledge
  Nominated for best paper [pdf]
  Yun-En Liu, Travis Mandel, Emma Brunskill, and Zoran Popovic.
  Computer Human Interaction (CHI) 2014.
• Offline Policy Evaluation Across Representations with Applications to Educational Games [pdf]
  Travis Mandel, Yun-En Liu, Sergey Levine, Emma Brunskill, Zoran Popovic.
  Autonomous Agents and Multi-Agent Systems (AAMAS) 2014.

2013

• Sequential Transfer in Multi-armed Bandit with Finite Set of Models [link]
  M.Azar, A.Lazaric and E.Brunskill
  NIPS 2013
• Regret Bounds for Reinforcement Learning with Policy Advice [link]
  M.Azar, A.Lazaric and E. Brunskill
  European Conference on Machine Learning (ECML) 2013
• Sample Complexity of Multi-task Reinforcement Learning. [pdf]
  E.Brunskill and L.Li
  Conference on Uncertainty in Artificial Intelligence (UAI) 2013
• Understanding Sequential Decisions via Inverse Reinforcement Learning.
  Siyuan Liu, Miguel Araujo, Emma Brunskill, Rosaldo Rossetti, Joao Barros, Ramayya Krishnan
  MDM 2013
• Predicting Player Moves in an Educational Game: A Hybrid Approach*
  Nominated for best paper [pdf]
  Yun-En Liu, Travis Mandel, Eric Butler, Erik Andersen, Eleanor O'Rourke, Emma Brunskill, Zoran Popovic.
  Educational Data Mining (EDM) 2013.
• Estimating Student Knowledge from Paired Interaction Data. [pdf]
  A.Rafferty, J.Davenport, and E.Brunskill
  Educational Data Mining (EDM) 2013
• New Potentials for Data-driven Intelligent Tutoring System Development and Optimization.
  Koedinger, K.R., Brunskill, E., Baker, R.S.J.d., McLaughlin, E.A., Stamper, J.
  AI Magazine
• Towards Operationalizing Outlier Detection in Community Health Programs
  T.McCarthy, B.DeRenzi, J.Blumenstock and E.Brunskill
  to appear as Note in Information Communications Technology for International Development (ICTD) 2013
• Analysis of the Impact of Errors Made During Health Data Collection Using Mobile Phones: Exploring Error Modeling and Automatic Diagnosis.
  S.Palkar and E.Brunskill
  ACM DEV 2013 (Poster)

2012

• Incentive Decision Processes [pdf]
  S.Reddi and E.Brunskill
  Conference on Uncertainty in Artificial Intelligence (UAI) 2012
• The Impact on Individualizing Student Models on Necessary Practice Opportunities
  *Nominated for best paper [pdf]
  J.I.Lee and E.Brunskill
  International Conference on Educational Data Mining (EDM) 2012
• Policy Building -- An Extension To User Modeling [pdf]
  M.Yudelson and E.Brunskill
  International Conference on Educational Data Mining (EDM) 2012
• Bayes-optimal reinforcement learning for discrete uncertainty domains.
  E.Brunskill
  AAMAS 2012 (Extended Abstract)
• Global and regional hearing impairment prevalence: an analysis of 42 studies in 29 countries [link]
  G.Stevens, S.Flaxman, E.Brunskill, M. Mascarenhas, C.Mathers, M.Finucane
  The European Journal of Public Health

2011 and prior

• Estimating prerequisite structure from noisy data [pdf]
  E.Brunskill
  International Conference on Educational Data Mining (EDM) 2011
• Partially observable sequential decision making for problem selection in an intelligent tutoring system [pdf]
  E.Brunskill and S.Russell
  poster in International Conference on Educational Data Mining (EDM) 2011
• Faster teaching by POMDP planning [pdf]
  A.Rafferty, E.Brunskill, T.Griffiths, and P.Shafto
  International Conference on Artificial Intelligence in Education (AIED) 2011
• Designing mobile interfaces for novice and low-literacy users [pdf]
  I. Medhi, S. Patnaik, E.Brunskill, S. N. Nagasena Gautama, W. Thies, and K. Toyama
  ACM Transactions on Computer-Human Interaction 2011
• Efficient planning under uncertainty with macro-actions [pdf]
  R.He, E. Brunskill and N. Roy
  Journal of Artificial Intelligence Research 2011
• Evaluating an adaptive multi-user educational tool for low-resource regions [pdf]
  E.Brunskill, S.Garg, C.Tseng, J.Pal and L.Findlater
  International Conference on Information and Communication Technologies and Development (ICTD) 2010
• RAPID: A reachable anytime planner for imprecisely-sensed domains [pdf]
  E.Brunskill and S.Russell
  Uncertainty in Artificial Intelligence (UAI) 2010
• PUMA: planning under uncertainty with macro-actions [pdf]
  R.He, E.Brunskill and N.Roy
  AAAI Conference on Artificial Intelligence (AAAI) 2010
• Planning in partially-observable switching-mode continuous domains [pdf]
  E.Brunskill, L.Kaelbling, T.Lozano-Perez, and N.Roy
  Annals of Mathematics and Artificial Intelligence 2010
• When policies can be trusted: analyzing a criteria to identify optimal policies in MDPs with unknown model parameters [pdf]
  (This corrected version makes a 2 sentence clarification that only the final DeltaQ represents an estimate of g)
  E.Brunskill
  International Conference on Automated Planning and Scheduling (ICAPS) 2010
• Provably efficient learning with typed parametric models [pdf]
  E.Brunskill, B. Leffler, L. Li, M. Littman, and N. Roy.
  Journal of Machine Learning Research 2009
• Evaluating the accuracy of data collection on mobile phones: a study of forms, SMS, and voice [pdf]
  S.Patnaik, E.Brunskill, and W.Thies.
  International Conference on Information and Communication Technologies and Development (ICTD) 2009
• Where to go: interpreting natural directions using global inference [pdf]
  Y.Wei, E.Brunskill, T.Kollar and N.Roy.
  International Conference on Robotics and Automation (ICRA) 2009
• CORL: a continuous-state offset-dynamics reinforcement learner [pdf]
  E.Brunskill, B.Leffler, L.Li, M.Littman and N.Roy.
  Uncertainty in Artificial Intelligence (UAI) 2008
  
• Perceptual switch rates with ambiguous structure-from-motion figures in bipolar disorder [pdf].
  K.Krug, E.Brunskill, A.Scarna, G.Goodwin and A.Parker.
  Proceedings of the Royal Society B 2008.
• A supervised learning approach for collision detection in legged locomotion. [pdf]
  F.Doshi, E.Brunskill, A.Shkolnik, T.Kollar, K.Rohanimanesh, R.Tedrake, and N.Roy
  International Conference on Intelligent Robots and Systems (IROS) 2007
• Topological mapping using spectral clustering and classification [pdf]
  E.Brunskill, T.Kollar and N.Roy
  International Conference on Intelligent Robots and Systems (IROS) 2007
• Adaptive state space construction with reinforcement learning for robots.
  E.Brunskill, E.Uchibe, and K.Doya
  poster International Conference on Robotics and Automation (ICRA) 2006
• SLAM using incremental probabilistic PCA and dimensionality reduction. [pdf]
  E.Brunskill and N.Roy.
  International Conference on Robotics and Automation (ICRA) 2005

Workshop and Symposium Papers

• Routing for rural health: optimizing community health worker visit schedules [pdf]
  E.Brunskill and N.Lesh
  position paper AAAI Spring Symposium on Artificial Intelligence for Development 2010
• Learning to identify locally actionable health anomalies [pdf]
  K.Chen, E.Brunskill, J.Dick and P.Dhadialla
  position paper AAAI Spring Symposium on Artificial Intelligence for Development 2010
• How close is close enough? Finding optimal policies in PAC-style reinforcement learning.
  E. Brunskill
  Abstract NIPS 2008 Workshop on Model Uncertainty and Risk in Reinforcement Learning
• Continuous-state POMDPs with hybrid dynamics. [pdf]
  E.Brunskill, L.Kaelbling, T.Lozano-Perez, and N.Roy.
  Proceedings of the International Symposium on Artificial Intelligence and Mathematics (ISAIM) 2008
• Continuous state POMDPs for object manipulation tasks.
  E.Brunskill
  Association for the Advancement of Artificial Intelligence (AAAI) 2007 Doctoral Consortium
• Lessons from prototyping a microfinance distance learning tool. [pdf]
  E.Brunskill and T.Parikh
  CHI workshop on User Centered Design for International Development 2007
• Building peer-to-peer systems with Chord, a distributed lookup service.
  F.Dabek, E.Brunskill, F.Kaashoek, D.Karger, R.Morris, I.Stoica, and H.Balakrishnan.
  in Proceedings of the 8th Workshop on Hot Topics in Operating Systems (HotOS-VIII) 2001

Ph.D. Thesis

• Compact Parametric Models for Efficient Sequential Decision Making in High-dimensional, Uncertain Domains MIT June 2009

Other Publications

• LittleDog learning locomotion project
  CSAIL research abstract 2006
• Impact of Bipolar Disorder on Ambiguous Structure-from-Motion Percepts
  E.Brunskill, et al.
  Society for Neuroscience Abstract (SFN) 2003