Timezone: »

 
Poster
Large-Scale Stochastic Sampling from the Probability Simplex
Jack Baker · Paul Fearnhead · Emily Fox · Christopher Nemeth

Thu Dec 06 07:45 AM -- 09:45 AM (PST) @ Room 210 #42
in Thu Poster Session A »

Stochastic gradient Markov chain Monte Carlo (SGMCMC) has become a popular method for scalable Bayesian inference. These methods are based on sampling a discrete-time approximation to a continuous time process, such as the Langevin diffusion. When applied to distributions defined on a constrained space the time-discretization error can dominate when we are near the boundary of the space. We demonstrate that because of this, current SGMCMC methods for the simplex struggle with sparse simplex spaces; when many of the components are close to zero. Unfortunately, many popular large-scale Bayesian models, such as network or topic models, require inference on sparse simplex spaces. To avoid the biases caused by this discretization error, we propose the stochastic Cox-Ingersoll-Ross process (SCIR), which removes all discretization error and we prove that samples from the SCIR process are asymptotically unbiased. We discuss how this idea can be extended to target other constrained spaces. Use of the SCIR process within a SGMCMC algorithm is shown to give substantially better performance for a topic model and a Dirichlet process mixture model than existing SGMCMC approaches.

Keywords: MCMC
[ Paper]  [ Poster] 

Author Information

Jack Baker (Lancaster University)
Paul Fearnhead (Lancaster University)

Paul Fearnhead is Professor of Statistics at Lancaster University. He received his DPhil in Statistics from the University of Oxford in 1998; was a postdoctoral researcher at the University of Oxford until 2001; and then moved to the University of Lancaster, initially as a Lecturer in Statistics. He has worked on Monte Carlo methods within Bayesian statistics, including applications in population genetics, changepoint detection and inference for diffusions. He was awarded the Royal Statistical Society's Guy medal in Bronze in 2007, and Cambridge University's Adams Prize in 2006.

Emily Fox (University of Washington, Apple)
Christopher Nemeth (Lancaster University)

More from the Same Authors