MITIGATING ISSUES CAUSED BY DEVICE DROPOUT IN KNOWLEDGE DISTILLATION BASED FEDERATED LEARNING

Federated learning (FL) is a machine learning technique where clients collaboratively train a machine learning model on distributed data re- siding on heterogeneous client devices while preserving data privacy. Applying knowledge distillation (KD) in FL have the potential to address its challenges such as device heterogeneity, communication issues, slow convergence and others. However, KD based FL system affected by client dropout and offline issue, due to resource constraints and unstable connectivity that poses challenge to the global model robustness and convergence. This research mitigates these issues by utilizing data similarity-based client clustering with battery aware client selection strategy to maintain robust global model performance under frequent device dropout. Clients are grouped according to data distribution similarity of their local data, that creates a redundancy pool for compensating dropout by substitute client from same cluster. Additionally, there is a battery aware algorithm that prioritizes devices with sufficient battery levels. Experimental evaluations done across MNIST dataset with non-IID data partitions across variety device energy profiles. In the simulation from our baseline system, our dropout affected system and mitigation strategy results show a 5 – 20 percentage point improvement in accuracy across 70 communication rounds. The findings contribute to advancing robust KD based FL system in resource constrained environments especially in Africa where power shortage exists.