Federated Learning via Spectral Methods with Compressed Communication and Local Steps

Federated Learning (FL) enables solving complex tasks over a large-scale network of edge devices with locally accumulated samples. Two key problems in FL are the communication bottleneck and data heterogeneity. While these challenges are well understood in classic settings, emerging paradigms bring them back as open problems. Recently, the spectral method Muon appeared as a superior competitor to Adam-like optimizers. Several works have attempted to transfer Muon to the FL setting; however, naive integration into classic schemes falls short. In particular, no prior work combines Muon with local steps and compression ~-- two key techniques of FL. In this paper, we develop a novel FL scheme and use it to close this gap. We provide rigorous convergence guarantees under mild assumptions, without assuming bounded heterogeneity. We validate our claims on an image-classification benchmark.