Timezone: »

 
Poster
Mean Field for the Stochastic Blockmodel: Optimization Landscape and Convergence Issues
Soumendu Sundar Mukherjee · Purnamrita Sarkar · Y. X. Rachel Wang · Bowei Yan

Wed Dec 05 02:00 PM -- 04:00 PM (PST) @ Room 210 #10
in Wed Poster Session B »

Variational approximation has been widely used in large-scale Bayesian inference recently, the simplest kind of which involves imposing a mean field assumption to approximate complicated latent structures. Despite the computational scalability of mean field, theoretical studies of its loss function surface and the convergence behavior of iterative updates for optimizing the loss are far from complete. In this paper, we focus on the problem of community detection for a simple two-class Stochastic Blockmodel (SBM). Using batch co-ordinate ascent (BCAVI) for updates, we give a complete characterization of all the critical points and show different convergence behaviors with respect to initializations. When the parameters are known, we show a significant proportion of random initializations will converge to ground truth. On the other hand, when the parameters themselves need to be estimated, a random initialization will converge to an uninformative local optimum.

Keywords: Variational Inference · Non-Convex Optimization

Author Information

Soumendu Sundar Mukherjee (University of California, Berkeley)
Purnamrita Sarkar (UT Austin)
Y. X. Rachel Wang (University of Sydney)
Bowei Yan (Jump Trading)

More from the Same Authors