Timezone: »

Approximation Based Variance Reduction for Reparameterization Gradients
Tomas Geffner · Justin Domke

Wed Dec 09 09:00 AM -- 11:00 AM (PST) @ Poster Session 3 #854

Flexible variational distributions improve variational inference but are harder to optimize. In this work we present a control variate that is applicable for any reparameterizable distribution with known mean and covariance, e.g. Gaussians with any covariance structure. The control variate is based on a quadratic approximation of the model, and its parameters are set using a double-descent scheme. We empirically show that this control variate leads to large improvements in gradient variance and optimization convergence for inference with non-factorized variational distributions.

Author Information

Tomas Geffner (UMass Amherst)
Justin Domke (University of Massachusetts, Amherst)

More from the Same Authors