Timezone: »

 
Poster
A Primal-Dual link between GANs and Autoencoders
Hisham Husain · Richard Nock · Robert Williamson

Tue Dec 10 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #118
in Deep Learning -- Generative Models »
Since the introduction of Generative Adversarial Networks (GANs) and Variational Autoencoders (VAE), the literature on generative modelling has witnessed an overwhelming resurgence. The impressive, yet elusive empirical performance of GANs has lead to the rise of many GAN-VAE hybrids, with the hopes of GAN level performance and additional benefits of VAE, such as an encoder for feature reduction, which is not offered by GANs. Recently, the Wasserstein Autoencoder (WAE) was proposed, achieving performance similar to that of GANs, yet it is still unclear whether the two are fundamentally different or can be further improved into a unified model. In this work, we study the $f$-GAN and WAE models and make two main discoveries. First, we find that the $f$-GAN and WAE objectives partake in a primal-dual relationship and are equivalent under some assumptions, which then allows us to explicate the success of WAE. Second, the equivalence result allows us to, for the first time, prove generalization bounds for Autoencoder models, which is a pertinent problem when it comes to theoretical analyses of generative models. Furthermore, we show that the WAE objective is related to other statistical quantities such as the $f$-divergence and in particular, upper bounded by the Wasserstein distance, which then allows us to tap into existing efficient (regularized) optimal transport solvers. Our findings thus present the first primal-dual relationship between GANs and Autoencoder models, comment on generalization abilities and make a step towards unifying these models.
Keywords: Learning Theory · Generative Models · Theory · Deep Learning -> Adversarial Networks; Deep Learning -> Deep Autoencoders; Deep Learning
[ Paper]  [ Poster] 

Author Information

Hisham Husain (The Australian National University)
Richard Nock (Data61, the Australian National University and the University of Sydney)
Robert Williamson (Australian National University & Data61)

More from the Same Authors