Stress-Testing Byzantine Defenses under Data Heterogeneity

Distributed learning is increasingly used for privacy-preserving training across decentralized datasets, but it remains vulnerable to Byzantine attacks, especially under realistic non-IID settings. In such scenarios, benign client gradients naturally diverge, making malicious updates harder to detect. While a few attacks have been designed for heterogeneous data, they are often impractical due to their reliance on gradient access or optimization overhead. In this work, we revisit classic IID-based attacks (ALIE, IPM) and show that, once calibrated for non-IID variance, they become significantly more effective, often outperforming specialized non-IID attacks like Min-Max and Min-Sum. Our results highlight that existing defenses can dramatically degrade under realistic threat models, calling for a reevaluation of current robustness claims in heterogeneous distributed learning.