Timezone: »

 
Gradient Descent: Robustness to Adversarial Corruption
Fu-Chieh Chang · Farhang Nabiei · Pei-Yuan Wu · Alexandru Cioba · Sattar Vakili · Alberto Bernacchia

@
in OPT 2022: Optimization for Machine Learning »
Event URL: https://openreview.net/forum?id=gNpieWW7xFR »

Optimization using gradient descent (GD) is a ubiquitous practice in various machine learning problems including training large neural networks. Noise-free GD and stochastic GD--corrupted by random noise--have been extensively studied in the literature, but less attention has been paid to an adversarial setting, that is subject to adversarial corruptions in the gradient values. In this work, we analyze the performance of GD under a proposed general adversarial framework. For the class of functions satisfying the Polyak-Łojasiewicz condition, we derive finite time bounds on a minimax optimization error. Based on this bound, we provide a guideline on the choice of learning rate sequence with theoretical guarantees on the robustness of GD against adversarial corruption.

Author Information

Fu-Chieh Chang (National Taiwan University)
Farhang Nabiei (University of Cambridge)
Pei-Yuan Wu (National Taiwan University)
Alexandru Cioba (Mediatek Research)
Sattar Vakili (MediaTek Research)
Alberto Bernacchia (MediaTek Research)

More from the Same Authors