Timezone: »

 
A Generalized and Distributable Generative Model for Private Representation Learning
Sheikh Shams Azam · Taejin Kim · Seyyedali Hosseinalipour · Carlee Joe-Wong · Saurabh Bagchi · Christopher Brinton

@
in Deep Generative Models and Downstream Applications »
Event URL: https://openreview.net/forum?id=cRKEnMKHY_z »

We study the problem of learning data representations that are private yet informative, i.e., providing information about intended "ally" targets while obfuscating sensitive "adversary" attributes. We propose a novel framework, Exclusion-Inclusion Generative Adversarial Network (EIGAN), that generalizes adversarial private representation learning (PRL) approaches to generate data encodings that account for multiple (possibly overlapping) ally and adversary targets. Preserving privacy is even more difficult when the data is collected across multiple distributed nodes, which for privacy reasons may not wish to share their data even for PRL training. Thus, learning such data representations at each node in a distributed manner (i.e., without transmitting source data) is of particular importance. This motivates us to develop D-EIGAN, the first distributed PRL method, based on fractional parameter sharing that promotes differentially private parameter sharing and also accounts for communication resource limitations. We theoretically analyze the behavior of adversaries under the optimal EIGAN and D-EIGAN encoders and consider the impact of dependencies among ally and adversary tasks on the encoder performance. Our experiments on real-world and synthetic datasets demonstrate the advantages of EIGAN encodings in terms of accuracy, robustness, and scalability; in particular, we show that EIGAN outperforms the previous state-of-the-art by a significant accuracy margin (47% improvement). The experiments further reveal that D-EIGAN's performance is consistent with EIGAN under different node data distributions and is resilient to communication constraints.

Author Information

Sheikh Shams Azam (Purdue University)
Taejin Kim (CMU, Carnegie Mellon University)
Seyyedali Hosseinalipour (Purdue University)
Carlee Joe-Wong (Carnegie Mellon University)
Saurabh Bagchi (Purdue University)

Saurabh Bagchi is a Professor in the School of Electrical and Computer Engineering and the Department of Computer Science at Purdue University in West Lafayette, Indiana. He is the founding Director of a university-wide resilience center at Purdue called CRISP (2017-present). He is the recipient of the Alexander von Humboldt Research Award (2018), the Adobe Faculty Award (2017), the AT&T Labs VURI Award (2016), the Google Faculty Award (2015), and the IBM Faculty Award (2014). He was elected to the IEEE Computer Society Board of Governors for the 2017-19 term and re-elected in 2019. He is an ACM Distinguished Scientist (2013), a Senior Member of IEEE (2007) and of ACM (2009), and a Distinguished Speaker for ACM (2012). He is a co-lead on the $39M WHIN-SMART center at Purdue. Saurabh's research interest is in dependable computing and distributed systems. He is proudest of the 21 PhD students and 50 Masters thesis students who have graduated from his research group and who are in various stages of building wonderful careers in industry or academia. In his group, he and his students have way too much fun building and breaking real systems. Along the way this has led to 10 best paper awards or nominations at IEEE/ACM conferences. Saurabh received his MS and PhD degrees from the University of Illinois at Urbana-Champaign and his BS degree from the Indian Institute of Technology Kharagpur, all in Computer Science. He was selected as the inaugural International Visiting Professor at IIT Kharagpur in 2018.

Christopher Brinton (Purdue University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors