Poster
Coupled Variational Bayes via Optimization Embedding
Bo Dai · Hanjun Dai · Niao He · Weiyang Liu · Zhen Liu · Jianshu Chen · Lin Xiao · Le Song

Wed Dec 5th 05:00 -- 07:00 PM @ Room 210 #11
in Wed Poster Session B »

Variational inference plays a vital role in learning graphical models, especially on large-scale datasets. Much of its success depends on a proper choice of auxiliary distribution class for posterior approximation. However, how to pursue an auxiliary distribution class that achieves both good approximation ability and computation efficiency remains a core challenge. In this paper, we proposed coupled variational Bayes which exploits the primal-dual view of the ELBO with the variational distribution class generated by an optimization procedure, which is termed optimization embedding. This flexible function class couples the variational distribution with the original parameters in the graphical models, allowing end-to-end learning of the graphical models by back-propagation through the variational distribution. Theoretically, we establish an interesting connection to gradient flow and demonstrate the extreme flexibility of this implicit distribution family in the limit sense. Empirically, we demonstrate the effectiveness of the proposed method on multiple graphical models with either continuous or discrete latent variables comparing to state-of-the-art methods.

Keywords: Graphical Models · Latent Variable Models · Variational Inference

Author Information

Bo Dai (Google Brain)
Hanjun Dai (Georgia Tech)
Niao He (UIUC)
Weiyang Liu (Georgia Institute of Technology)
Zhen Liu (Georgia Institute of Technology)
Jianshu Chen (Tencent AI Lab)
Lin Xiao (Microsoft Research)
Le Song (Ant Financial & Georgia Institute of Technology)

More from the Same Authors