AI Lunch and Seminar

Fish Tung - Adversarial Inversion: Self-supervision with Adversarial Priors

Abstract: We as humans form explanations of visual observations in terms of familiar concepts and memories that are used to interpret and complete information of the image pixels. Computer Vision researchers have developed excellent methods that learn a direct mapping from images to desired outputs using human annotations or synthetically generated data. Despite their success, such supervised models very much depend on the amount of annotated data available, a gap we seek to address.

In this talks, we introduce adversarial inversion, a weakly supervised neural network model that combines self-supervision with adversarial constraints. Given visual input, our model first generates a set of desirable intermediate latent variables, which we call “imaginations”, e.g., 3D pose and camera viewpoint, such that these imagination matches what we observe. Adversarial inversion can be trained with or without paired supervision of standard supervised models, as it does not require paired annotations. It can instead exploit a large number of unlabelled images. We empirically show adversarial inversion outperforms previous state-of-the-art supervised models on 3D human pose estimation and 3D scene depth estimation. Further, we show interesting results on biased image editing.

Joint work with Adam Harley, William Seto and Katerina Fragkiadaki