홈 › Papers

Sparta: Spatially Attentive and Adversarially Robust Activations

Adversarial training has been demonstrated to be useful for improving the robustness of deep neural networks (DNNs). However, the impacts of basic network components (e.g., ReLU, the widely used activation function for DNNs) to adversarial training effectiveness received less attention and has not been comprehensively investigated so far. To fill this gap, in this work, we argue that the spatially-shared and input-independent activating properties of the ReLU make the DNNs under both standard training and adversarial training less robust to white-box adversarial attacks. To address such challenges, we design a novel activation function, i.e., Sparta: Spatially Attentive and Adversarially Robust Activation, which enables DNNs to achieve higher robustness (i.e., lower error rate on adversarial examples) and accuracy (i.e., lower error rate on clean examples) than the DNNs based on the state-of-the-art activation functions. We further investigate the relationships between our Sparta and the state-of-the-art search-based activation function, i.e., Swish, and feature denoising method, providing insights about the advantages of our method. Moreover, comprehensive evaluations have demonstrated two important properties of our method: First, superior transferability across DNNs. Our adversarially trained Sparta function for one DNN (e.g., ResNet-18) can be fixed to train another adversarially robust DNN (e.g., ResNet-34), achieving higher robustness than the one using vanilla ReLU as activation. Second, superior transferability across datasets. The Sparta function trained on one dataset (e.g., CIFAR-10) can be employed to train adversarially robust DNNs on another dataset (e.g., SVHN) and helps achieve higher robustness than DNNs with vanilla ReLU as activation. These properties have highlighted the flexibility and versatility of Sparta. Accompanying code is also submitted.

Code (0)

등록된 구현이 없습니다.

Tasks

Methods 이 논문이 사용한 방법론

Similar Papers 제목 키워드 기반