Timezone: »
We propose a novel sampling framework for inference in probabilistic models: an active learning approach that converges more quickly (in wall-clock time) than Markov chain Monte Carlo (MCMC) benchmarks. The central challenge in probabilistic inference is numerical integration, to average over ensembles of models or unknown (hyper-)parameters (for example to compute marginal likelihood or a partition function). MCMC has provided approaches to numerical integration that deliver state-of-the-art inference, but can suffer from sample inefficiency and poor convergence diagnostics. Bayesian quadrature techniques offer a model-based solution to such problems, but their uptake has been hindered by prohibitive computation costs. We introduce a warped model for probabilistic integrands (likelihoods) that are known to be non-negative, permitting a cheap active learning scheme to optimally select sample locations. Our algorithm is demonstrated to offer faster convergence (in seconds) relative to simple Monte Carlo and annealed importance sampling on both synthetic and real-world examples.
Author Information
Tom Gunter (Apple)
Michael A Osborne (U Oxford)
Roman Garnett (Washington University in St. Louis)
Philipp Hennig (University of Tübingen and MPI Tübingen)
Stephen J Roberts (University of Oxford)
More from the Same Authors
-
2020 Poster: Effective Diversity in Population Based Reinforcement Learning »
Jack Parker-Holder · Aldo Pacchiano · Krzysztof M Choromanski · Stephen J Roberts -
2020 Poster: Gaussian Process Bandit Optimization of the Thermodynamic Variational Objective »
Vu Nguyen · Vaden Masrani · Rob Brekelmans · Michael A Osborne · Frank Wood -
2020 Spotlight: Effective Diversity in Population Based Reinforcement Learning »
Jack Parker-Holder · Aldo Pacchiano · Krzysztof M Choromanski · Stephen J Roberts -
2020 Poster: Explicit Regularisation in Gaussian Noise Injections »
Alexander Camuto · Matthew Willetts · Umut Simsekli · Stephen J Roberts · Chris C Holmes -
2020 Poster: Bayesian Optimization for Iterative Learning »
Vu Nguyen · Sebastian Schulze · Michael A Osborne -
2020 Poster: Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits »
Jack Parker-Holder · Vu Nguyen · Stephen J Roberts -
2016 Workshop: Optimizing the Optimizers »
Maren Mahsereci · Alex Davies · Philipp Hennig -
2016 Poster: Bayesian Optimization for Probabilistic Programs »
Thomas Rainforth · Tuan Anh Le · Jan-Willem van de Meent · Michael A Osborne · Frank Wood -
2015 Workshop: Probabilistic Integration »
Michael A Osborne · Philipp Hennig -
2015 Symposium: Algorithms Among Us: the Societal Impacts of Machine Learning »
Michael A Osborne · Adrian Weller · Murray Shanahan -
2015 Poster: Probabilistic Line Searches for Stochastic Optimization »
Maren Mahsereci · Philipp Hennig -
2015 Poster: Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees »
François-Xavier Briol · Chris Oates · Mark Girolami · Michael A Osborne -
2015 Oral: Probabilistic Line Searches for Stochastic Optimization »
Maren Mahsereci · Philipp Hennig -
2015 Spotlight: Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees »
François-Xavier Briol · Chris Oates · Mark Girolami · Michael A Osborne -
2014 Poster: Incremental Local Gaussian Regression »
Franziska Meier · Philipp Hennig · Stefan Schaal -
2014 Poster: Probabilistic ODE Solvers with Runge-Kutta Means »
Michael Schober · David Duvenaud · Philipp Hennig -
2014 Oral: Probabilistic ODE Solvers with Runge-Kutta Means »
Michael Schober · David Duvenaud · Philipp Hennig -
2013 Workshop: Bayesian Optimization in Theory and Practice »
Matthew Hoffman · Jasper Snoek · Nando de Freitas · Michael A Osborne · Ryan Adams · Sebastien Bubeck · Philipp Hennig · Remi Munos · Andreas Krause -
2013 Poster: The Randomized Dependence Coefficient »
David Lopez-Paz · Philipp Hennig · Bernhard Schölkopf -
2013 Poster: Σ-Optimality for Active Learning on Gaussian Random Fields »
Yifei Ma · Roman Garnett · Jeff Schneider -
2012 Workshop: Probabilistic Numerics »
Philipp Hennig · John P Cunningham · Michael A Osborne -
2012 Poster: Active Learning of Model Evidence Using Bayesian Quadrature »
Michael A Osborne · David Duvenaud · Roman Garnett · Carl Edward Rasmussen · Stephen J Roberts · Zoubin Ghahramani -
2011 Workshop: Bayesian optimization, experimental design and bandits: Theory and applications »
Nando de Freitas · Roman Garnett · Frank R Hutter · Michael A Osborne -
2011 Poster: Optimal Reinforcement Learning for Gaussian Systems »
Philipp Hennig -
2006 Poster: Bayesian Image Super-resolution, Continued »
Lyndsey C Pickup · David Capel · Stephen J Roberts · Andrew Zisserman -
2006 Spotlight: Bayesian Image Super-resolution, Continued »
Lyndsey C Pickup · David Capel · Stephen J Roberts · Andrew Zisserman