Timezone: »

 
Spotlight
Fast Black-box Variational Inference through Stochastic Trust-Region Optimization
Jeffrey Regier · Michael Jordan · Jon McAuliffe

Wed Dec 06 11:25 AM -- 11:30 AM (PST) @ Hall A
in Theory, Probabilistic Methods »

We introduce TrustVI, a fast second-order algorithm for black-box variational inference based on trust-region optimization and the reparameterization trick. At each iteration, TrustVI proposes and assesses a step based on minibatches of draws from the variational distribution. The algorithm provably converges to a stationary point. We implement TrustVI in the Stan framework and compare it to ADVI. TrustVI typically converges in tens of iterations to a solution at least as good as the one that ADVI reaches in thousands of iterations. TrustVI iterations can be more computationally expensive, but total computation is typically an order of magnitude less in our experiments.

Author Information

Jeffrey Regier (UC Berkeley)
Michael Jordan (UC Berkeley)
Jon McAuliffe (UC Berkeley)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors