Timezone: »

 
Poster
On gradient regularizers for MMD GANs
Michael Arbel · Danica J. Sutherland · Mikołaj Bińkowski · Arthur Gretton

Tue Dec 04 07:45 AM -- 09:45 AM (PST) @ Room 210 #21
in Tue Poster Session A »
We propose a principled method for gradient-based regularization of the critic of GAN-like models trained by adversarially optimizing the kernel of a Maximum Mean Discrepancy (MMD). We show that controlling the gradient of the critic is vital to having a sensible loss function, and devise a method to enforce exact, analytical gradient constraints at no additional cost compared to existing approximate techniques based on additive regularizers. The new loss function is provably continuous, and experiments show that it stabilizes and accelerates training, giving image generation models that outperform state-of-the art methods on $160 \times 160$ CelebA and $64 \times 64$ unconditional ImageNet.
Keywords: Kernel Methods · Adversarial Networks · Generative Models
[ Paper]  [ Poster] 

Author Information

Michael Arbel (UCL)
Danica J. Sutherland (Gatsby Unit, UCL)

D.J. Sutherland is a Research Assistant Professor at TTI-Chicago, and will begin as an Assistant Professor in UBC Computer Science in 2021. D.J. received a PhD from Carnegie Mellon in 2016, and was a postdoc at the Gatsby Unit, UCL, from 2016-19. D.J.’s research is largely in the connection between kernel methods and deep learning, with a focus on interfacing theory with methodology in areas including two-sample testing, generative models, distribution regression, and representation learning.

Mikołaj Bińkowski (Imperial College London)
Arthur Gretton (Gatsby Unit, UCL)

Arthur Gretton is a Professor with the Gatsby Computational Neuroscience Unit at UCL. He received degrees in Physics and Systems Engineering from the Australian National University, and a PhD with Microsoft Research and the Signal Processing and Communications Laboratory at the University of Cambridge. He previously worked at the MPI for Biological Cybernetics, and at the Machine Learning Department, Carnegie Mellon University. Arthur's recent research interests in machine learning include the design and training of generative models, both implicit (e.g. GANs) and explicit (high/infinite dimensional exponential family models), nonparametric hypothesis testing, and kernel methods. He has been an associate editor at IEEE Transactions on Pattern Analysis and Machine Intelligence from 2009 to 2013, an Action Editor for JMLR since April 2013, an Area Chair for NeurIPS in 2008 and 2009, a Senior Area Chair for NeurIPS in 2018, an Area Chair for ICML in 2011 and 2012, and a member of the COLT Program Committee in 2013. Arthur was program chair for AISTATS in 2016 (with Christian Robert), tutorials chair for ICML 2018 (with Ruslan Salakhutdinov), workshops chair for ICML 2019 (with Honglak Lee), program chair for the Dali workshop in 2019 (with Krikamol Muandet and Shakir Mohammed), and co-organsier of the Machine Learning Summer School 2019 in London (with Marc Deisenroth).

More from the Same Authors