Timezone: »

 
Poster
Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model
Erik Nijkamp · Mitch Hill · Song-Chun Zhu · Ying Nian Wu

Tue Dec 10 05:30 PM -- 07:30 PM (PST) @ East Exhibition Hall B + C #84
in Deep Learning -- Generative Models »

This paper studies a curious phenomenon in learning energy-based model (EBM) using MCMC. In each learning iteration, we generate synthesized examples by running a non-convergent, non-mixing, and non-persistent short-run MCMC toward the current model, always starting from the same initial distribution such as uniform noise distribution, and always running a fixed number of MCMC steps. After generating synthesized examples, we then update the model parameters according to the maximum likelihood learning gradient, as if the synthesized examples are fair samples from the current model. We treat this non-convergent short-run MCMC as a learned generator model or a flow model. We provide arguments for treating the learned non-convergent short-run MCMC as a valid model. We show that the learned short-run MCMC is capable of generating realistic images. More interestingly, unlike traditional EBM or MCMC, the learned short-run MCMC is capable of reconstructing observed images and interpolating between images, like generator or flow models. The code can be found in the Appendix.

Keywords: MCMC · Generative Models · Deep Learning · Probabilistic Methods
[ Paper]  [ Poster] 

Author Information

Erik Nijkamp (UCLA)
Mitch Hill (UCLA Department of Statistics)
Song-Chun Zhu (UCLA)
Ying Nian Wu (University of California, Los Angeles)

More from the Same Authors