Timezone: »

 
Regularizing Score-based Models with Score Fokker-Planck Equations
Chieh-Hsin Lai · Yuhta Takida · Naoki Murata · Toshimitsu Uesaka · Yuki Mitsufuji · Stefano Ermon

@
in NeurIPS 2022 Workshop on Score-Based Methods »
Event URL: https://openreview.net/forum?id=WqW7tC32v8N »

Score-based generative models learn a family of noise-conditional score functions corresponding to the data density perturbed with increasingly large amounts of noise. These pertubed data densities are tied together by the Fokker-Planck equation (FPE), a PDE governing the spatial-temporal evolution of a density undergoing a diffusion process. In this work, we derive a corresponding equation characterizing the noise-conditional scores of the perturbed data densities (i.e., their gradients), termed the score FPE. Surprisingly, despite impressive empirical performance, we observe that scores learned via denoising score matching (DSM) do not satisfy the underlying score FPE. We mathematically analyze two implications of satisfying the score FPE and a potential explanation for why the score FPE is not satisfied in practice. At last, we propose to regularize the DSM objective to enforce satisfaction of the score FPE, and show its effectiveness on synthetic data and MNIST.

Author Information

Chieh-Hsin Lai (Sony Group Corporation)
Yuhta Takida (Sony Corporation)
Naoki Murata (Sony Group Corporation)
Toshimitsu Uesaka (Sony Group Corporation)
Yuki Mitsufuji (Sony Group Corporation)
Stefano Ermon (Stanford)

More from the Same Authors