Timezone: »
Federated Learning allows the training of machine learning models by using the computation and private data resources of many distributed clients. Most existing results on Federated Learning (FL) assume the clients have ground-truth labels. However, in many practical scenarios, clients may be unable to label task-specific data due to a lack of expertise or resource. We propose SemiFL to address the problem of combining communication-efficient FL such as FedAvg with Semi-Supervised Learning (SSL). In SemiFL, clients have completely unlabeled data and can train multiple local epochs to reduce communication costs, while the server has a small amount of labeled data. We provide a theoretical understanding of the success of data augmentation-based SSL methods to illustrate the bottleneck of a vanilla combination of communication-efficient FL with SSL. To address this issue, we propose alternate training to 'fine-tune global model with labeled data' and 'generate pseudo-labels with the global model.' We conduct extensive experiments and demonstrate that our approach significantly improves the performance of a labeled server with unlabeled clients training with multiple local epochs. Moreover, our method outperforms many existing SSFL baselines and performs competitively with the state-of-the-art FL and SSL results.
Author Information
Enmao Diao (Duke University)
I am a fourth-year Ph.D. candidate advised by Prof. Vahid Tarokh in Electrical Engineering at Duke University, Durham, North Carolina, USA. I was born in Chengdu, Sichuan, China in 1994. I received the B.S. degree in Computer Science and Electrical Engineering from Georgia Institute of Technology, Georgia, USA, in 2016 and the M.S. degree in Electrical Engineering from Harvard University, Cambridge, USA, in 2018.
Jie Ding (University of Minnesota)
Vahid Tarokh (Duke University)
More from the Same Authors
-
2021 : Benchmarking Data-driven Surrogate Simulators for Artificial Electromagnetic Materials »
Yang Deng · Juncheng Dong · Simiao Ren · Omar Khatib · Mohammadreza Soltani · Vahid Tarokh · Willie Padilla · Jordan Malof -
2022 : Building Large Machine Learning Models from Small Distributed Models: A Layer Matching Approach »
xinwei zhang · Bingqing Song · Mehrdad Honarkhah · Jie Ding · Mingyi Hong -
2022 : PerFedSI: A Framework for Personalized Federated Learning with Side Information »
Liam Collins · Enmao Diao · Tanya Roosta · Jie Ding · Tao Zhang -
2022 Spotlight: Self-Aware Personalized Federated Learning »
Huili Chen · Jie Ding · Eric W. Tramel · Shuang Wu · Anit Kumar Sahu · Salman Avestimehr · Tao Zhang -
2022 Poster: Self-Aware Personalized Federated Learning »
Huili Chen · Jie Ding · Eric W. Tramel · Shuang Wu · Anit Kumar Sahu · Salman Avestimehr · Tao Zhang -
2022 Poster: GAL: Gradient Assisted Learning for Decentralized Multi-Organization Collaborations »
Enmao Diao · Jie Ding · Vahid Tarokh -
2022 Poster: Inference and Sampling for Archimax Copulas »
Yuting Ng · Ali Hasan · Vahid Tarokh -
2020 Poster: Assisted Learning: A Framework for Multi-Organization Learning »
Xun Xian · Xinran Wang · Jie Ding · Reza Ghanadan -
2020 Spotlight: Assisted Learning: A Framework for Multi-Organization Learning »
Xun Xian · Xinran Wang · Jie Ding · Reza Ghanadan -
2019 Poster: Gradient Information for Representation and Modeling »
Jie Ding · Robert Calderbank · Vahid Tarokh -
2019 Poster: SpiderBoost and Momentum: Faster Variance Reduction Algorithms »
Zhe Wang · Kaiyi Ji · Yi Zhou · Yingbin Liang · Vahid Tarokh -
2018 Poster: Learning Bounds for Greedy Approximation with Explicit Feature Maps from Multiple Kernels »
Shahin Shahrampour · Vahid Tarokh