Timezone: »

 
Poster
Robust Binary Models by Pruning Randomly-initialized Networks
Chen Liu · Ziqi Zhao · Sabine Süsstrunk · Mathieu Salzmann

Thu Dec 01 02:00 PM -- 04:00 PM (PST) @ Hall J #124
in Poster Session 6 »

Robustness to adversarial attacks was shown to require a larger model capacity, and thus a larger memory footprint. In this paper, we introduce an approach to obtain robust yet compact models by pruning randomly-initialized binary networks. Unlike adversarial training, which learns the model parameters, we initialize the model parameters as either +1 or −1, keep them fixed, and find a subnetwork structure that is robust to attacks. Our method confirms the Strong Lottery Ticket Hypothesis in the presence of adversarial attacks, and extends this to binary networks. Furthermore, it yields more compact networks with competitive performance than existing works by 1) adaptively pruning different network layers; 2) exploiting an effective binary initialization scheme; 3) incorporating a last batch normalization layer to improve training stability. Our experiments demonstrate that our approach not only always outperforms the state-of-the-art robust binary networks, but also can achieve accuracy better than full-precision ones on some datasets. Finally, we show the structured patterns of our pruned binary networks.

Keywords: model compression · Adversarial Robustness
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Chen Liu (City University of Hong Kong)
Ziqi Zhao (École polytechnique fédérale de Lausanne (EPFL))
Ziqi Zhao

Ziqi Zhao is currently a master student in EPFL. His research interests are adversarial robustness, network pruning, network quantization and indoor localization.

Sabine Süsstrunk (EPFL)
Mathieu Salzmann (EPFL)

More from the Same Authors