Timezone: »
Poster
Approximation Based Variance Reduction for Reparameterization Gradients
Tomas Geffner · Justin Domke
Flexible variational distributions improve variational inference but are harder to optimize. In this work we present a control variate that is applicable for any reparameterizable distribution with known mean and covariance, e.g. Gaussians with any covariance structure. The control variate is based on a quadratic approximation of the model, and its parameters are set using a double-descent scheme. We empirically show that this control variate leads to large improvements in gradient variance and optimization convergence for inference with non-factorized variational distributions.
Author Information
Tomas Geffner (UMass Amherst)
Justin Domke (University of Massachusetts, Amherst)
More from the Same Authors
-
2023 Poster: Provable convergence guarantees for black-box variational inference »
Justin Domke · Robert Gower · Guillaume Garrigos -
2023 Poster: Discriminative Calibration »
Yuling Yao · Justin Domke -
2021 Poster: MCMC Variational Inference via Uncorrected Hamiltonian Annealing »
Tomas Geffner · Justin Domke -
2021 Poster: Amortized Variational Inference for Simple Hierarchical Models »
Abhinav Agrawal · Justin Domke -
2020 Poster: Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization »
Abhinav Agrawal · Daniel Sheldon · Justin Domke -
2019 Poster: Thompson Sampling and Approximate Inference »
My Phan · Yasin Abbasi Yadkori · Justin Domke -
2019 Poster: Provable Gradient Variance Guarantees for Black-Box Variational Inference »
Justin Domke -
2019 Poster: Divide and Couple: Using Monte Carlo Variational Objectives for Posterior Approximation »
Justin Domke · Daniel Sheldon -
2019 Spotlight: Divide and Couple: Using Monte Carlo Variational Objectives for Posterior Approximation »
Justin Domke · Daniel Sheldon -
2018 Poster: Using Large Ensembles of Control Variates for Variational Inference »
Tomas Geffner · Justin Domke -
2018 Poster: Importance Weighting and Variational Inference »
Justin Domke · Daniel Sheldon