Timezone: »
Boltzmann machines are powerful distributions that have been shown to be an effective prior over binary latent variables in variational autoencoders (VAEs). However, previous methods for training discrete VAEs have used the evidence lower bound and not the tighter importance-weighted bound. We propose two approaches for relaxing Boltzmann machines to continuous distributions that permit training with importance-weighted bounds. These relaxations are based on generalized overlapping transformations and the Gaussian integral trick. Experiments on the MNIST and OMNIGLOT datasets show that these relaxations outperform previous discrete VAEs with Boltzmann priors. An implementation which reproduces these results is available.
Author Information
Arash Vahdat (Quadrant.ai (D-Wave))
Evgeny Andriyash (D-Wave)
William Macready (D-Wave)
More from the Same Authors
-
2022 Workshop: NeurIPS 2022 Workshop on Score-Based Methods »
Yingzhen Li · Yang Song · Valentin De Bortoli · Francois-Xavier Briol · Wenbo Gong · Alexia Jolicoeur-Martineau · Arash Vahdat -
2021 Poster: A Contrastive Learning Approach for Training Variational Autoencoder Priors »
Jyoti Aneja · Alex Schwing · Jan Kautz · Arash Vahdat -
2021 Poster: Score-based Generative Modeling in Latent Space »
Arash Vahdat · Karsten Kreis · Jan Kautz -
2021 Poster: Controllable and Compositional Generation with Latent-Space Energy-Based Models »
Weili Nie · Arash Vahdat · Anima Anandkumar -
2021 Poster: Don’t Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence »
Tianshi Cao · Alex Bie · Arash Vahdat · Sanja Fidler · Karsten Kreis -
2020 Poster: NVAE: A Deep Hierarchical Variational Autoencoder »
Arash Vahdat · Jan Kautz -
2020 Spotlight: NVAE: A Deep Hierarchical Variational Autoencoder »
Arash Vahdat · Jan Kautz -
2020 Poster: On the distance between two neural networks and the stability of learning »
Jeremy Bernstein · Arash Vahdat · Yisong Yue · Ming-Yu Liu -
2017 Poster: Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks »
Arash Vahdat