Skip to yearly menu bar Skip to main content


Phase Transition from Clean Training to Adversarial Training

Yue Xing · Qifan Song · Guang Cheng

Hall J (level 1) #206

Keywords: [ adversarial training ] [ Adversarial Robustness ]


Adversarial training is one important algorithm to achieve robust machine learning models. However, numerous empirical results show a great performance degradation from clean training to adversarial training (e.g., 90+\% vs 67\% testing accuracy on CIFAR-10 dataset), which does not match the theoretical guarantee delivered by the existing studies. Such a gap inspires us to explore the existence of an (asymptotic) phase transition phenomenon with respect to the attack strength: adversarial training is as well behaved as clean training in the small-attack regime, but there is a sharp transition from clean training to adversarial training in the large-attack regime. We validate this conjecture in linear regression models, and conduct comprehensive experiments in deep neural networks.

Chat is not available.