Timezone: »

 
Poster
Sinkhorn Natural Gradient for Generative Models
Zebang Shen · Zhenfu Wang · Alejandro Ribeiro · Hamed Hassani

Thu Dec 10 09:00 PM -- 11:00 PM (PST) @ Poster Session 6 #1852
in Poster Session 7 » 
We consider the problem of minimizing a functional over a parametric family of probability measures, where the parameterization is characterized via a push-forward structure. 
An important application of this problem is in training generative adversarial networks.  
In this regard, we propose a novel Sinkhorn Natural Gradient (SiNG) algorithm which acts as a steepest descent method on the probability space endowed with the Sinkhorn divergence.
We show that the Sinkhorn information matrix (SIM), a key component of SiNG, has an explicit expression and can be evaluated accurately in complexity that scales logarithmically with respect to the desired accuracy. This is in sharp contrast to  existing natural gradient methods that can only be carried out approximately.
Moreover, in practical applications when only Monte-Carlo type integration is available, we design an empirical estimator for SIM and provide the stability analysis.
In our experiments, we quantitatively compare SiNG with state-of-the-art SGD-type solvers on generative tasks to demonstrate its efficiency and efficacy of our method.
Keywords: Algorithms -> Online Learning; Theory -> Computational Complexity; Theory -> Information Theory · Probabilistic Methods -> Bayesian Nonparametrics

Author Information

Zebang Shen (University of Pennsylvania)
Zhenfu Wang (Peking University)
Alejandro Ribeiro (University of Pennsylvania)
Hamed Hassani (UPenn)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors