Timezone: »

 
Poster
Don’t Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
Tianshi Cao · Alex Bie · Arash Vahdat · Sanja Fidler · Karsten Kreis

Thu Dec 09 08:30 AM -- 10:00 AM (PST) @
in Poster Session 6 »

Although machine learning models trained on massive data have led to breakthroughs in several areas, their deployment in privacy-sensitive domains remains limited due to restricted access to data. Generative models trained with privacy constraints on private data can sidestep this challenge, providing indirect access to private data instead. We propose DP-Sinkhorn, a novel optimal transport-based generative method for learning data distributions from private data with differential privacy. DP-Sinkhorn minimizes the Sinkhorn divergence, a computationally efficient approximation to the exact optimal transport distance, between the model and data in a differentially private manner and uses a novel technique for controlling the bias-variance trade-off of gradient estimates. Unlike existing approaches for training differentially private generative models, which are mostly based on generative adversarial networks, we do not rely on adversarial objectives, which are notoriously difficult to optimize, especially in the presence of noise imposed by privacy constraints. Hence, DP-Sinkhorn is easy to train and deploy. Experimentally, we improve upon the state-of-the-art on multiple image modeling benchmarks and show differentially private synthesis of informative RGB images.

Keywords: Machine Learning · Generative Model · Privacy · Optimal Transport

Author Information

Tianshi Cao (University of Toronto)
Alex Bie (University of Waterloo)
Arash Vahdat (NVIDIA Research)
Sanja Fidler (University of Toronto)
Karsten Kreis (NVIDIA)

More from the Same Authors