Hoda Heidari

I am the K&L Gates Career Development Assistant Professor in Ethics and Computational Technologies at Carnegie Mellon University with joint appointments in the Machine Learning Department and the Institute for Software, Systems, and Society. I am also affiliated with the Human-Computer Interaction Institute and the Heinz College of Information Systems and Public Policy at CMU, and I co-lead the university-wide Responsible AI Initiative and the K&L Gates Initiative for Ethics and Computational Technologies.

I am broadly interested in the Ethical and Societal Aspects of Artificial Intelligence and Machine Learning. In particular, my research has addressed issues of Fairness and Accountability. For more information, please take a look at my bio, CV (last updated Sep 2024), Google Scholar profile, and the Research section of this page.

My work has been generously supported by the NSF Program on Fairness in AI in Collaboration with Amazon, PwC, CyLab, Meta, and J. P. Morgan. I am a senior personnel at AI-SDM: the NSF AI Institute for Societal Decision Making.

---

Team

I currently advise the following excellent doctoral students and postdocs (alphabetical order):

• Cyrus Cousins (co-supervised with Jana Schaich Borg, Vincent Conitzer, and Walter Sinnott-Armstrong)
• Michael Feffer (co-advised with Zack Lipton)
• Samsara Foubert
• Nari Johnson
• Vijay Keswani (co-supervised with Jana Schaich Borg, Vincent Conitzer, and Walter Sinnott-Armstrong)
• Rachel Kim (co-advised with Rayid Ghani)
• Blaine Kuehnert
• Megan Li (co-advised with Lorrie Cranor)
• Tori Qiu (co-advised with Norman Sadeh)
• Anusha Sinha (co-advised with Steven Wu)
• Ningjing Tang (co-advised with Hong Shen)
• Kimberly Truong (co-advised with Steven Wu)
• Rebecca Yu

---

Research Papers

• 'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants
  Shivani Kapania, William Agnew, Motahhare Eslami*, Hoda Heidari*, Sarah Fox*
  Preprint, 2024.


• Public Procurement for Responsible AI? Understanding U.S. Cities' Practices, Challenges, and Needs
  Nari Johnson, Elise Silva, Harrison Leon, Motahhare Eslami, Beth Schwanke, Ravit Dotan, Hoda Heidari
  Preprint, 2024.


• Red-Teaming for Generative AI: Silver Bullet or Security Theater?
  Michael Feffer, Anusha Sinha, Zachary C. Lipton, Hoda Heidari
  The AAAI /ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2024. Best Paper Award at AIES.


• On the Pros and Cons of Active Learning for Moral Preference Elicitation
  Vijay Keswani, Vincent Conitzer*, Hoda Heidari*, Jana Schaich Borg*, and Walter Sinnott-Armstrong*
  The AAAI /ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2024.


• On The Stability of Moral Preferences: A Problem with Computational Elicitation Methods
  Kyle Boerstler, Vijay Keswani, Lok Chan, Jana Schaich Borg, Vincent Conitzer, Hoda Heidari, and Walter Sinnott-Armstrong
  The AAAI /ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2024.


• The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment
  Nari Johnson, Sanika Moharana, Christina Harrington, Nazanin Andalibi, Hoda Heidari*, Motahhare Eslami*.
  The ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2024.


• AI Failure Cards: Understanding and Supporting Grassroots Efforts to Mitigate AI Failures in Homeless Services
  Ningjing Tang, Jiayin Zhi, Tzu-Sheng Kuo, Calla Kainaroi, Jeremy J. Northup, Kenneth Holstein, Haiyi Zhu, Hoda Heidari, and Hong Shen.
  The ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2024.


• Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use
  Anna Kawakami, Amanda Coston, Hoda Heidari*, Kenneth Holstein*, and Haiyi Zhu*.
  The ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW), 2024.


• The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder Early- stage Deliberations Around Public Sector AI Proposals
  Anna Kawakami, Amanda Coston, Haiyi Zhu*, Hoda Heidari*, Kenneth Holstein*.
  The ACM CHI conference on Human Factors in Computing Systems (CHI), 2024.


• Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models
  Benjamin Laufer, Jon Kleinberg*, and Hoda Heidari*
  The Web Conference, 2024.


• Beneficent Intelligence: A Capability Approach to Modeling Benefit, Assistance, and Associated Moral Failures through AI Systems
  J. London* and H. Heidari*
  Minds and Machines (to appear), 2024.


• A Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementarity
  Charvi Rastogi*, Leqi Liu*, Ken Holstein, Hoda Heidari
  The AAAI Conference on Human Computation and Crowdsourcing (HCOMP), 2023.


• The AI Incident Database as an Educational Tool to Raise Awareness of Harms: A Classroom Exploration of Efficacy, Limitations, & Future Design Improvements
  M. Feffer, N. Martelaro, and H. Heidari
  ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), 2023.


• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
  Emily Black, Rakshit Naidu, Rayid Ghani, Kit T. Rodolfa, Daniel E. Ho, and Hoda Heidari
  ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), 2023.


• Informational Diversity and Affinity Bias in Team Formation Dynamics
  Hoda Heidari, Solon Barocas, Karen Levy, and Jon Kleinberg.
  ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), 2023.


• Perspectives on Incorporating Expert Feedback into Model Updates
  Valerie Chen, Umang Bhatt, Hoda Heidari, Adrian Weller, and Ameet Talwalkar
  Pattern (Volume 4, Issue 7), 2023.


• From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research
  M. Feffer, M. Skirpan, Z. Lipton*, and H. Heidari*
  The AAAI /ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2023.


• A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms
  A. Coston, A. Kawakami, H. Zhu, K. Holstein, and H. Heidari
  IEEE Conference on Secure and Trustworthy Machine Learning (SAT-ML), 2023. Winner of a Best Paper Award at SAT-ML.


• Local Justice & ML: Modeling and Inferring Dynamic Ethical Judgments Around High-stakes Allocations
  V. Chen, J. Williams, D, Leben, and H. Heidari
  The AAAI Conference on Artificial Intelligence (AAAI), 2023.


• Moral Machine or Tyranny of the Majority?
  M. Feffer, H. Heidari*, and Z. Lipton*
  The AAAI Conference on Artificial Intelligence (AAAI), 2023.


• Bayesian Persuasion for Algorithmic Recourse
  K. Harris, V. Chen, J. S. Kim, A. Talwalkar, H. Heidari, and S. Wu
  Neural and Information Processing Systems (NeurIPS), 2022.


• Four Years of FAccT: A Reflexive, Mixed-Methods Analysis of Research Contributions, Shortcomings, and Future Prospects
  B. Laufer, S. Jain, A.F. Cooper, J. Kleinberg, and H. Heidari
  The ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2022.


• Strategic Instrumental Variable Regression: Recovering Causal Relationships From Strategic Responses
  K. Harris, Daniel Ngo*, Logan Stapleton*, H. Heidari, and S. Wu
  The International Conference on Machine Learning (ICML), 2022.


• Stateful Strategic Regression
  K. Harris, H. Heidari, and S. Wu
  Neural and Information Processing Systems (NeurIPS), 2021.


• On Modeling Human Perceptions of Allocation Policies with Uncertain Outcomes
  H. Heidari, S. Barocas, J. Kleinberg, and K. Levy
  The ACM Conference on Economics and Computation (EC), 2021. Winner of an Exemplary Track Award at EC.


• Addressing the Long-term Impact of ML Decisions via Policy Regret
  D. Lindner, H. Heidari, and A. Krause
  The International Joint Conference on Artificial Intelligence (IJCAI), 2021.


• Fair equality of chances: fairness for statistical prediction-based decision-making
  M. Loi, A. Herlitz, and H. Heidari
  AAAI /ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2021.


• A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairness
  M. Yaghini, A. Krause, and H. Heidari
  AAAI /ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2021.


• Allocating Opportunities in a Dynamic Model of Intergenerational Mobility
  H. Heidari, J. Kleinberg
  The ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2021. Winner of a Best Paper Award at FAccT.


• On the Desiderata for Online Altruism: Nudging for Equitable Donations
  N. Mota, A. Chakraborty, A. J. Biega, K. P. Gummadi, H. Heidari
  The ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW), 2020


• Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning
  M. Srivastava, H. Heidari, A. Krause
  The International Conference on Knowledge Discovery and Data Mining (KDD), 2019


• On the Long-term Impact of Algorithmic Decision Policies: Effort Unfairness and Feature Segregation through Social Learning
  H. Heidari, V. Nanda, K. P. Gummadi
  The International Conference on Machine Learning (ICML), 2019


• A Moral Framework for Understanding of Fair ML through Economic Models of Equality of Opportunity
  H. Heidari, M. Loi, K. P. Gummadi, A. Krause
  The ACM Conference on Fairness, Accountability, and Transparency (ACM FAT*), 2019


• On the Impact of Choice Architectures on Inequality in Online Donation Platforms
  A. Chakraborty, N. Mota, A. J. Biega, K. P. Gummadi, H. Heidari
  The Web Conference (WWW), 2019


• Fairness Behind a Veil of Ignorance: A Welfare Analysis for Automated Decision Making
  H. Heidari, C. Ferrari, K. P. Gummadi, A. Krause
  Neural and Information Processing Systems (NeurIPS), 2018


• A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual and Group Unfairness via Inequality Indices
  T. Speicher, H. Heidari, N. Grgic-Hlaca, K. P. Gummadi, A. Singla, A. Weller, M. B. Zafar
  The International Conference on Knowledge Discovery and Data Mining (KDD), 2018


• Fairness in Criminal Justice Risk Assessments: The State of the Art
  R. Berk, H. Heidari, S. Jabbari, M. Kearns, A. Roth
  Sociological Methods and Research, 2018


• Preventing Disparate Treatment in Sequential Decision Making
  H. Heidari, A. Krause
  The International Joint Conference on Artificial Intelligence (IJCAI), 2018


• A Convex Framework for Fair Regression
  R. Berk, H. Heidari, S. Jabbari, M. Joseph, M. Kearns, J. Morgenstern, S. Neel, A. Roth
  In FAT-ML Workshop, 2017


• Pricing a Low-regret Seller
  H. Heidari, M. Mahdian, U. Syed, S. Vassilvistskii, S. Yazdanbod
  The International Conference on Machine Learning (ICML), 2016


• Tight Policy Regret Bounds for Improving and Decaying Bandits
  H. Heidari, M. Kearns, A. Roth
  The International Joint Conference on Artificial Intelligence (IJCAI), 2016


• Integrating Market Makers, Limit Orders, and Continuous Trade in Prediction Markets
  H. Heidari, S. Lahaie, D. Pennock, J. W. Vaughan
  The ACM Conference on Economics and Computation (EC), 2015


• Competitive contagion in networks
  S. Goyal, H. Heidari, M. Kearns
  Games and Economic Behavior, Elsevier, 2014


• Learning from Contagion (Without Timestamps)
  K. Amin, H. Heidari, M. Kearns
  The International Conference on Machine Learning (ICML), 2014


• New Models for Competitive Contagion
  M. Draief, H. Heidari, M. Kearns
  The AAAI Conference on Artificial Intelligence (AAAI), 2014


• Depth-Workload Tradeoffs for Workforce Organization
  H. Heidari, M. Kearns
  The Conference on Human Computation & Crowdsourcing (HCOMP), 2013

---

Teaching

• Responsible AI (10-735): Spring 2024, Fall 2024
• Machine Learning, Ethics, and Society (10-613/713): Spring 2023
• Mathematical foundations of ML (10-606): Fall 2022
• Computational foundations of ML (10-607): Fall 2022
• Probabilistic Graphical Models (10-708): Spring 2022
• Machine Learning, Ethics, and Society (10-613/713): Fall 2021
• Fairness, Explainability, and Accountability for Machine Learning (10-712): Spring 2019, Fall 2020

---

Selected Service

• Organizer of CMU’s expert convening on Supporting NIST’s Development of Guidelines on Red-teaming for Generative AI, 2024
• Co-organizer of Evaluating Generative AI Systems: the Good, the Bad, and the Hype.
  Sponsored by the K&L Gates Initiative for Ethics and Computational Technologies at CMU. <>2024
• Chair and organizer of the 2022 PI Meeting for the NSF Program on Fairness in AI in Collaboration with Amazon, 2022
• Tutorial Co-chair for the ACM FAccT conference, 2022
• Co-organizer of the ICLR Workshop on Responsible AI, 2021
• Co-organizer of the NeurIPS Workshop on Human-centric Machine Learning, co-organizer 2019
• Co-organizer of the Tutorial on Economic Theories of Distributive Justice for Fair M at WWW, 2019
• Senior Program Committee for FAccT 2022, EC 2022, AAAI 2022, NeurIPS 2021, ICML 2021