AI Seminar sponsored by Fortive

Han Zhao -- Costs and Benefits of Invariant Representation Learning

Abstract: The success of supervised machine learning in recent years crucially hinges on the availability of large-scale and unbiased data. However, it is often time-consuming and expensive to collect such data. Recent advances in deep learning focus on learning invariant representations that have found abundant applications in both domain adaptation and algorithmic fairness. However, it is not clear what price we have to pay in terms of task utility for such universal representations. In this talk, I will discuss my recent work on understanding and learning invariant representations.

In the first part, I will focus on understanding the costs of existing invariant representations by characterizing a fundamental tradeoff between invariance and utility. In particular, I will use domain adaptation as an example to both theoretically and empirically show such tradeoff in achieving small joint generalization error. This result also implies that when the base rates differ, any fair algorithm has to make a large error on at least one of the groups.

In the second part of the talk, I will focus on designing learning algorithms to escape the existing tradeoff and to utilize the benefits of invariant representations. I will show how the algorithm can be used to ensure equalized treatment of individuals between groups, and what additional problem structure that permits efficient domain adaptation through learning invariant representations.

Bio: Han Zhao is a final-year PhD student at the Machine Learning Department, Carnegie Mellon University. At CMU, he works with Prof. Geoff Gordon. Before coming to CMU, he obtained his BEng degree from the Computer Science Department at Tsinghua University and MMath from the University of Waterloo. He has a broad interest in both the theoretical and applied side of machine learning. In particular, he works on invariant representation learning, probabilistic reasoning with Sum-Product Networks, transfer and multitask learning, and computational social choice. More details are here: https://www.cs.cmu.edu/~hzhao1/