`

Timezone: »

 
VAEs meet Diffusion Models: Efficient and High-Fidelity Generation
Kushagra Pandey · Avideep Mukherjee · Piyush Rai · Abhishek Kumar
Event URL: https://openreview.net/forum?id=-J8dM4ed_92 »

Diffusion Probabilistic models have been shown to generate state-of-the-art results on several competitive image synthesis benchmarks but lack a low-dimensional, interpretable latent space, and are slow at generation. On the other hand, Variational Autoencoders (VAEs) have access to a low-dimensional latent space but, despite recent advances, exhibit poor sample quality. We present VAEDM, a novel generative framework for \textit{refining} VAE generated samples using diffusion models while also presenting a novel conditional forward process parameterization for diffusion models. We show that the resulting parameterization can improve upon the unconditional diffusion model in terms of sampling efficiency during inference while also equipping diffusion models with the low-dimensional VAE inferred latent code. Furthermore, we show that the proposed model exhibits out-of-the-box capabilities for downstream tasks like image superresolution and denoising.

Author Information

Kushagra Pandey (Indian Institute of Technology, Kanpur)

I am a graduate student in the Computer Science Department at IIT Kanpur. My current research interests are scalable bayesian inference and probabilistic deep generative models and their downstream applications to computer vision and computational genomics.

Avideep Mukherjee (Indian Institute of Technology Kanpur)
Piyush Rai (IIT Kanpur)
Abhishek Kumar (Google)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors

  • 2021 : NeurInt-Learning Interpolation by Neural ODEs »
    Avinandan Bose · Aniket Das · Yatin Dandi · Piyush Rai
  • 2021 Poster: CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks »
    Sakshi Varshney · Vinay Kumar Verma · P. K. Srijith · Lawrence Carin · Piyush Rai
  • 2020 Poster: Calibrating CNNs for Lifelong Learning »
    Pravendra Singh · Vinay Kumar Verma · Pratik Mazumder · Lawrence Carin · Piyush Rai
  • 2019 : Poster session »
    Sebastian Farquhar · Erik Daxberger · Andreas Look · Matt Benatan · Ruiyi Zhang · Marton Havasi · Fredrik Gustafsson · James A Brofos · Nabeel Seedat · Micha Livne · Ivan Ustyuzhaninov · Adam Cobb · Felix D McGregor · Patrick McClure · Tim R. Davidson · Gaurush Hiranandani · Sanjeev Arora · Mikhal Itkina · Didrik Nielsen · William Harvey · Matias Valdenegro-Toro · Stefano Peluchetti · Riccardo Moriconi · Tianyu Cui · Vaclav Smidl · Taylan Cemgil · Jack Fitzsimons · He Zhao · · mariana vargas vieyra · Apratim Bhattacharyya · Rahul Sharma · Geoffroy Dubourg-Felonneau · Jonathan Warrell · Slava Voloshynovskiy · Mihaela Rosca · Jiaming Song · Andrew Ross · Homa Fashandi · Ruiqi Gao · Hooshmand Shokri Razaghi · Joshua Chang · Zhenzhong Xiao · Vanessa Boehm · Giorgio Giannone · Ranganath Krishnan · Joe Davison · Arsenii Ashukha · Jeremiah Liu · Sicong (Sheldon) Huang · Evgenii Nikishin · Sunho Park · Nilesh Ahuja · Mahesh Subedar · · Artyom Gadetsky · Jhosimar Arias Figueroa · Tim G. J. Rudner · Waseem Aslam · Adrián Csiszárik · John Moberg · Ali Hebbal · Kathrin Grosse · Pekka Marttinen · Bang An · Hlynur Jónsson · Samuel Kessler · Abhishek Kumar · Mikhail Figurnov · Omesh Tickoo · Steindor Saemundsson · Ari Heljakka · Dániel Varga · Niklas Heim · Simone Rossi · Max Laves · Waseem Gharbieh · Nicholas Roberts · Luis Armando Pérez Rey · Matthew Willetts · Prithvijit Chakrabarty · Sumedh Ghaisas · Carl Shneider · Wray Buntine · Kamil Adamczewski · Xavier Gitiaux · Suwen Lin · Hao Fu · Gunnar Rätsch · Aidan Gomez · Erik Bodin · Dinh Phung · Lennart Svensson · Juliano Tusi Amaral Laganá Pinto · Milad Alizadeh · Jianzhun Du · Kevin Murphy · Beatrix Benkő · Shashaank Vattikuti · Jonathan Gordon · Christopher Kanan · Sontje Ihler · Darin Graham · Michael Teng · Louis Kirsch · Tomas Pevny · Taras Holotyak
  • 2018 Poster: Delta-encoder: an effective sample synthesis method for few-shot object recognition »
    Eli Schwartz · Leonid Karlinsky · Joseph Shtok · Sivan Harary · Mattias Marder · Abhishek Kumar · Rogerio Feris · Raja Giryes · Alex Bronstein
  • 2018 Spotlight: Delta-encoder: an effective sample synthesis method for few-shot object recognition »
    Eli Schwartz · Leonid Karlinsky · Joseph Shtok · Sivan Harary · Mattias Marder · Abhishek Kumar · Rogerio Feris · Raja Giryes · Alex Bronstein
  • 2018 Poster: Co-regularized Alignment for Unsupervised Domain Adaptation »
    Abhishek Kumar · Prasanna Sattigeri · Kahini Wadhawan · Leonid Karlinsky · Rogerio Feris · Bill Freeman · Gregory Wornell
  • 2017 : Poster session + Coffee break »
    Mikael Kågebäck · Igor Melnyk · Amir-Hossein Karimi · Gino Brunner · Ershad Banijamali · Chris Donahue · Jake Zhao · Giambattista Parascandolo · Valentin Thomas · Abhishek Kumar · Chris Burgess · Amanda Nilsson · Maria Larsson · Cian Eastwood · Momchil Peychev
  • 2017 Poster: Semi-supervised Learning with GANs: Manifold Invariance with Improved Inference »
    Abhishek Kumar · Prasanna Sattigeri · Tom Fletcher
  • 2012 Poster: Simultaneously Leveraging Output and Task Structures for Multiple-Output Regression »
    Piyush Rai · Abhishek Kumar · Hal Daumé III
  • 2011 Poster: Co-regularized Multi-view Spectral Clustering »
    Abhishek Kumar · Piyush Rai · Hal Daumé III
  • 2010 Poster: Co-regularization Based Semi-supervised Domain Adaptation »
    Hal Daumé III · Abhishek Kumar · Avishek Saha