Timezone: »

 
Conditional Moment Alignment for Improved Generalization in Federated Learning
Jayanth Reddy Regatti · Songtao Lu · Abhishek Gupta · Ness Shroff

Fri Dec 02 07:20 AM -- 07:27 AM (PST) @
in Federated Learning: Recent Advances and New Challenges »
Event URL: https://openreview.net/forum?id=EBTbXqqqoFg »

In this work, we study model heterogeneous Federated Learning (FL) for classification where different clients have different model architectures. Unlike existing works on model heterogeneity, we neither require access to a public dataset nor do we impose constraints on the model architecture of clients and ensure that the clients' models and data are private. We prove a generalization result, that provides fundamental insights into the role of the representations in FL and propose a theoretically grounded algorithm \textbf{Fed}erated \textbf{C}onditional \textbf{M}oment \textbf{A}lignment (\pap) that aligns class conditional distributions of each client in the feature space. We prove the convergence and empirically, we show that \pap outperforms other baselines on CIFAR-10, MNIST, EMNIST, FEMNIST in the considered setting.

Author Information

Jayanth Reddy Regatti (The Ohio State University)

I am a PhD student, researching on robustness in distributed machine learning and Federated Learning. This summer I interned at Microsoft BingAds, where I worked on unsupervised representation learning.

Songtao Lu (IBM Thomas J. Watson Research Center)
Abhishek Gupta (Ohio State University)
Ness Shroff (The Ohio State University)

More from the Same Authors