Timezone: »

Differentially Private Model Personalization
Prateek Jain · John Rush · Adam Smith · Shuang Song · Abhradeep Guha Thakurta

We study personalization of supervised learning with user-level differential privacy. Consider a setting with many users, each of whom has a training data set drawn from their own distribution $P_i$. Assuming some shared structure among the problems $P_i$, can users collectively learn the shared structure---and solve their tasks better than they could individually---while preserving the privacy of their data? We formulate this question using joint, user-level differential privacy---that is, we control what is leaked about each user's entire data set. We provide algorithms that exploit popular non-private approaches in this domain like the Almost-No-Inner-Loop (ANIL) method, and give strong user-level privacy guarantees for our general approach. When the problems $P_i$ are linear regression problems with each user's regression vector lying in a common, unknown low-dimensional subspace, we show that our efficient algorithms satisfy nearly optimal estimation error guarantees. We also establish a general, information-theoretic upper bound via an exponential mechanism-based algorithm.

Author Information

Prateek Jain (Google Research)
John Rush (Google)

I come from a pure mathematics background, formerly a harmonic analyst and mathematical physicist. I transferred to machine learning on the software side after grad school, and joined Google in 2018, working on federated learning. I am a main author of TensorFlow Federated; ask me about it!

Adam Smith (Boston University)
Shuang Song (Google)

I am currently a 6th year PhD student in [UC San Diego](http://www.cs.ucsd.edu/). I am working with [Prof. Kamalika Chaudhuri](http://cseweb.ucsd.edu/~kamalika/) in Machine Learning and Differential Privacy. Before joining UCSD, I obtained my BSc degree in Mathematics and Computer Science from [The Hong Kong University of Science and Technology](http://www.ust.hk). I was an intern in the [Google Brain Team](https://research.google.com/teams/brain/) during Summer 2017.

Abhradeep Guha Thakurta (Google Research - Brain Team)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors