Timezone: »

 
Poster
TRS: Transferability Reduced Ensemble via Promoting Gradient Diversity and Model Smoothness
Zhuolin Yang · Linyi Li · Xiaojun Xu · Shiliang Zuo · Qian Chen · Pan Zhou · Benjamin Rubinstein · Ce Zhang · Bo Li

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @
in Poster Session 1 »

Adversarial Transferability is an intriguing property - adversarial perturbation crafted against one model is also effective against another model, while these models are from different model families or training processes. To better protect ML systems against adversarial attacks, several questions are raised: what are the sufficient conditions for adversarial transferability, and how to bound it? Is there a way to reduce the adversarial transferability in order to improve the robustness of an ensemble ML model? To answer these questions, in this work we first theoretically analyze and outline sufficient conditions for adversarial transferability between models; then propose a practical algorithm to reduce the transferability between base models within an ensemble to improve its robustness. Our theoretical analysis shows that only promoting the orthogonality between gradients of base models is not enough to ensure low transferability; in the meantime, the model smoothness is an important factor to control the transferability. We also provide the lower and upper bounds of adversarial transferability under certain conditions. Inspired by our theoretical analysis, we propose an effective Transferability Reduced Smooth (TRS) ensemble training strategy to train a robust ensemble with low transferability by enforcing both gradient orthogonality and model smoothness between base models. We conduct extensive experiments on TRS and compare with 6 state-of-the-art ensemble baselines against 8 whitebox attacks on different datasets, demonstrating that the proposed TRS outperforms all baselines significantly.

Keywords: Robustness · Adversarial Robustness and Security

Author Information

Zhuolin Yang (Shanghai Jiao Tong University)
Linyi Li (University of Illinois at Urbana-Champaign)

A Ph.D. candidate working on robust machine learning and verification.

Xiaojun Xu (University of Illinois, Urbana Champaign)
Shiliang Zuo (University of Illinois Urbana-Champaign)
Qian Chen (Tencent)
Pan Zhou
Benjamin Rubinstein (University of Melbourne)
Ce Zhang (ETH Zurich)
Bo Li (UIUC)

More from the Same Authors