Scaleable input gradient regularization for adversarial robustness
In this work we revisit gradient regularization for adversarial robustness with some new ingredients. First, we derive new per-image theoretical robustness bounds based on local gradient information. These bounds strongly motivate input gradient regularization. Second, we implement a scaleable version of input gradient regularization which avoids double backpropagation: adversarially robust ImageNet models are trained in 33 hours on four consumer grade GPUs. Finally, we show experimentally and through theoretical certification that input gradient regularization is competitive with adversarial training. Moreover we demonstrate that gradient regularization does not lead to gradient obfuscation or gradient masking.
Code (1)
Tasks
Adversarial AttackAdversarial DefenseAdversarial RobustnessSimilar Papers 제목 키워드 기반
Lipschitz regularized Deep Neural Networks generalize and are adversarially robust
In this work we study input gradient regularization of deep neural networks, and demonstrate that such regularization leads to generalization proofs and improved adversarial robustness. The proof of generalization does n…
Adversarial RobustnessInput Hessian Regularization of Neural Networks
Regularizing the input gradient has shown to be effective in promoting the robustness of neural networks. The regularization of the input's Hessian is therefore a natural next step. A key challenge here is the computatio…
Adversarial AttackUnderstanding and Combating Robust Overfitting via Input Loss Landscape Analysis and Regularization
Adversarial training is widely used to improve the robustness of deep neural networks to adversarial attack. However, adversarial training is prone to overfitting, and the cause is far from clear. This work sheds light o…
Adversarial AttackDefense Against Adversarial Attacks on No-Reference Image Quality Models with Gradient Norm Regularization
The task of No-Reference Image Quality Assessment (NR-IQA) is to estimate the quality score of an input image without additional information. NR-IQA models play a crucial role in the media industry, aiding in performance…
Adversarial RobustnessImage Quality AssessmentNo-Reference Image Quality AssessmentRethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off
Adversarial training often suffers from a robustness-accuracy trade-off, where achieving high robustness comes at the cost of accuracy. One approach to mitigate this trade-off is leveraging invariance regularization, whi…
Adversarial DefenseKnowledge DistillationRepresentation LearningSelf-Supervised Learning