TANGO-MoE — When Can Federated Mixture-of-Experts Trust Transfer History?
Abstract
Federated mixture-of-experts (MoE) training combines updates from clients with different data distributions, compute budgets, and locally specialized expert identities. Historical transfer outcomes offer a low-cost coordination signal only when candidate transfers are distinguishable within the available measurement budget and remain informative across rounds. We introduce TANGO-MoE, an exact-edge framework that represents each candidate as a client-specific directed transfer and measures its source-local and macro-recipient effects through controlled server interventions. Across three heterogeneity regimes and three schedule perturbations, the constrained stable between-edge variance component is estimated at zero at the reference intervention dose. Resolving a median edge pair at 5% ranking error requires 22.7 independent probes per edge, compared with three available. Lag-one edge rankings are nearly uncorrelated, and only 22.0% of jointly beneficial edges remain beneficial in the next round. Although historical scores contain a weak but detectable ranking signal, all 16 leakage-free selectors yield negative mean next-round selected utility. These findings motivate a reliability-first design in which history prioritizes measurement and controls abstention, current edge-specific evidence determines admissibility, and fresh composition-level evidence is reserved for deployment certification.