Timezone: »
It is difficult to use subsampling with variational inference in hierarchical models since the number of local latent variables scales with the dataset. Thus, inference in hierarchical models remains a challenge at a large scale. It is helpful to use a variational family with a structure matching the posterior, but optimization is still slow due to the huge number of local distributions. Instead, this paper suggests an amortized approach where shared parameters simultaneously represent all local distributions. This approach is similarly accurate as using a given joint distribution (e.g., a full-rank Gaussian) but is feasible on datasets that are several orders of magnitude larger. It is also dramatically faster than using a structured variational distribution.
Author Information
Abhinav Agrawal (UMass Amherst)
Ph.D. student working to scale probabilistic inference.
Justin Domke (University of Massachusetts, Amherst)
More from the Same Authors
-
2023 Poster: Provable convergence guarantees for black-box variational inference »
Justin Domke · Robert Gower · Guillaume Garrigos -
2023 Poster: Discriminative Calibration »
Yuling Yao · Justin Domke -
2021 Poster: MCMC Variational Inference via Uncorrected Hamiltonian Annealing »
Tomas Geffner · Justin Domke -
2020 Poster: Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization »
Abhinav Agrawal · Daniel Sheldon · Justin Domke -
2020 Poster: Approximation Based Variance Reduction for Reparameterization Gradients »
Tomas Geffner · Justin Domke -
2019 Poster: Thompson Sampling and Approximate Inference »
My Phan · Yasin Abbasi Yadkori · Justin Domke -
2019 Poster: Provable Gradient Variance Guarantees for Black-Box Variational Inference »
Justin Domke -
2019 Poster: Divide and Couple: Using Monte Carlo Variational Objectives for Posterior Approximation »
Justin Domke · Daniel Sheldon -
2019 Spotlight: Divide and Couple: Using Monte Carlo Variational Objectives for Posterior Approximation »
Justin Domke · Daniel Sheldon -
2018 Poster: Using Large Ensembles of Control Variates for Variational Inference »
Tomas Geffner · Justin Domke -
2018 Poster: Importance Weighting and Variational Inference »
Justin Domke · Daniel Sheldon