Benchmarking Adversarial Robustness
Deep neural networks are vulnerable to adversarial examples, which becomes one of the most important research problems in the development of deep learning. While a lot of efforts have been made in recent years, it is of great significance to perform correct and complete evaluations of the adversarial attack and defense algorithms. In this paper, we establish a comprehensive, rigorous, and coherent benchmark to evaluate adversarial robustness on image classification tasks. After briefly reviewing plenty of representative attack and defense methods, we perform large-scale experiments with two robustness curves as the fair-minded evaluation criteria to fully understand the performance of these methods. Based on the evaluation results, we draw several important findings and provide insights for future research.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial AttackAdversarial RobustnessBenchmarkingimage-classificationImage ClassificationSimilar Papers 제목 키워드 기반
RL-Based Method for Benchmarking the Adversarial Resilience and Robustness of Deep Reinforcement Learning Policies
This paper investigates the resilience and robustness of Deep Reinforcement Learning (DRL) policies to adversarial perturbations in the state space. We first present an approach for the disentanglement of vulnerabilities…
BenchmarkingDeep Reinforcement LearningDisentanglementreinforcement-learning+4Guardians of Image Quality: Benchmarking Defenses Against Adversarial Attacks on Image Quality Metrics
In the field of Image Quality Assessment (IQA), the adversarial robustness of the metrics poses a critical concern. This paper presents a comprehensive benchmarking study of various defense mechanisms in response to the …
Adversarial AttackAdversarial PurificationAdversarial RobustnessBenchmarking+1VISAT: Benchmarking Adversarial and Distribution Shift Robustness in Traffic Sign Recognition with Visual Attributes
We present VISAT, a novel open dataset and benchmarking suite for evaluating model robustness in the task of traffic sign recognition with the presence of visual attributes. Built upon the Mapillary Traffic Sign Dataset …
Traffic Sign RecognitionMulti-Task LearningAdversarial AttackAutonomous DrivingA Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking
The robustness of deep neural networks is usually lacking under adversarial examples, common corruptions, and distribution shifts, which becomes an important research problem in the development of deep learning. Although…
Adversarial RobustnessBenchmarkingimage-classificationImage Classification+1Benchmarking the Spatial Robustness of DNNs via Natural and Adversarial Localized Corruptions
The robustness of DNNs is a crucial factor in safety-critical applications, particularly in complex and dynamic environments where localized corruptions can arise. While previous studies have evaluated the robustness of …
BenchmarkingSegmentationSemantic Segmentation