Timezone: »
Sampling from posterior distributions using Markov chain Monte Carlo (MCMC) methods can require an exhaustive number of iterations, particularly when the posterior is multi-modal as the MCMC sampler can become trapped in a local mode for a large number of iterations. In this paper, we introduce the pseudo-extended MCMC method as a simple approach for improving the mixing of the MCMC sampler for multi-modal posterior distributions. The pseudo-extended method augments the state-space of the posterior using pseudo-samples as auxiliary variables. On the extended space, the modes of the posterior are connected, which allows the MCMC sampler to easily move between well-separated posterior modes. We demonstrate that the pseudo-extended approach delivers improved MCMC sampling over the Hamiltonian Monte Carlo algorithm on multi-modal posteriors, including Boltzmann machines and models with sparsity-inducing priors.
Author Information
Christopher Nemeth (Lancaster University)
Fredrik Lindsten (Linköping University)
Maurizio Filippone (EURECOM)
James Hensman (PROWLER.io)
More from the Same Authors
-
2022 : A Modelling Framework for Catalysing Progress in the Rod-Shaped Bacterial Cell Growth Discourse »
Shashi Nagarajan · Fredrik Lindsten -
2023 Poster: One-Line-of-Code Data Mollification Improves Optimization of Likelihood-based Generative Models »
Ba-Hien Tran · Giulio Franzese · Pietro Michiardi · Maurizio Filippone -
2023 Poster: Continuous-Time Functional Diffusion Processes »
Giulio Franzese · Giulio Corallo · Simone Rossi · Markus Heinonen · Maurizio Filippone · Pietro Michiardi -
2023 Poster: Learning Rate Free Bayesian Inference in Constrained Domains »
Louis Sharrock · Lester Mackey · Christopher Nemeth -
2022 Poster: All You Need is a Good Functional Prior for Bayesian Deep Learning »
Ba-Hien Tran · Simone Rossi · Dimitrios Milios · Maurizio Filippone -
2021 Poster: Model Selection for Bayesian Autoencoders »
Ba-Hien Tran · Simone Rossi · Dimitrios Milios · Pietro Michiardi · Edwin Bonilla · Maurizio Filippone -
2020 Poster: Walsh-Hadamard Variational Inference for Bayesian Deep Learning »
Simone Rossi · Sebastien Marmin · Maurizio Filippone -
2020 Poster: Markovian Score Climbing: Variational Inference with KL(p||q) »
Christian Naesseth · Fredrik Lindsten · David Blei -
2019 Poster: Calibration tests in multi-class classification: A unifying framework »
David Widmann · Fredrik Lindsten · Dave Zachariah -
2019 Spotlight: Calibration tests in multi-class classification: A unifying framework »
David Widmann · Fredrik Lindsten · Dave Zachariah -
2019 Poster: Parameter elimination in particle Gibbs sampling »
Anna Wigren · Riccardo Sven Risuleo · Lawrence Murray · Fredrik Lindsten -
2019 Oral: Parameter elimination in particle Gibbs sampling »
Anna Wigren · Riccardo Sven Risuleo · Lawrence Murray · Fredrik Lindsten -
2018 Poster: Gaussian Process Conditional Density Estimation »
Vincent Dutordoir · Hugh Salimbeni · James Hensman · Marc Deisenroth -
2018 Poster: Infinite-Horizon Gaussian Processes »
Arno Solin · James Hensman · Richard Turner -
2018 Poster: Large-Scale Stochastic Sampling from the Probability Simplex »
Jack Baker · Paul Fearnhead · Emily Fox · Christopher Nemeth -
2018 Poster: Dirichlet-based Gaussian Processes for Large-scale Calibrated Classification »
Dimitrios Milios · Raffaello Camoriano · Pietro Michiardi · Lorenzo Rosasco · Maurizio Filippone -
2018 Poster: Learning Invariances using the Marginal Likelihood »
Mark van der Wilk · Matthias Bauer · ST John · James Hensman -
2017 Poster: Convolutional Gaussian Processes »
Mark van der Wilk · Carl Edward Rasmussen · James Hensman -
2017 Oral: Convolutional Gaussian Processes »
Mark van der Wilk · Carl Edward Rasmussen · James Hensman -
2017 Poster: Identification of Gaussian Process State Space Models »
Stefanos Eleftheriadis · Tom Nicholson · Marc Deisenroth · James Hensman -
2015 Poster: MCMC for Variationally Sparse Gaussian Processes »
James Hensman · Alexander Matthews · Maurizio Filippone · Zoubin Ghahramani -
2013 Workshop: Probabilistic Models for Big Data »
Neil D Lawrence · Joaquin Quiñonero-Candela · Tianshi Gao · James Hensman · Zoubin Ghahramani · Max Welling · David Blei · Ralf Herbrich -
2013 Session: Tutorial Session A »
James Hensman -
2012 Poster: Fast Variational Inference in the Conjugate Exponential Family »
James Hensman · Magnus Rattray · Neil D Lawrence