Skip to yearly menu bar Skip to main content


Poster

Invariant Representations without Adversarial Training

Daniel Moyer · Shuyang Gao · Rob Brekelmans · Aram Galstyan · Greg Ver Steeg

Room 517 AB #120

Keywords: [ Information Theory ] [ Variational Inference ] [ Deep Autoencoders ] [ Adversarial Networks ] [ Representation Learning ] [ Fairness, Accountability, and Transparency ]


Abstract:

Representations of data that are invariant to changes in specified factors are useful for a wide range of problems: removing potential biases in prediction problems, controlling the effects of covariates, and disentangling meaningful factors of variation. Unfortunately, learning representations that exhibit invariance to arbitrary nuisance factors yet remain useful for other tasks is challenging. Existing approaches cast the trade-off between task performance and invariance in an adversarial way, using an iterative minimax optimization. We show that adversarial training is unnecessary and sometimes counter-productive; we instead cast invariant representation learning as a single information-theoretic objective that can be directly optimized. We demonstrate that this approach matches or exceeds performance of state-of-the-art adversarial approaches for learning fair representations and for generative modeling with controllable transformations.

Live content is unavailable. Log in and register to view live content