Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Unlike recent robustness research, this benchmark evaluates performance on commonplace corruptions not worst-case adversarial corruptions. We find that there are negligible changes in relative corruption robustness from AlexNet to ResNet classifiers, and we discover ways to enhance corruption robustness. Then we propose a new dataset called Icons-50 which opens research on a new kind of robustness, surface variation robustness. With this dataset we evaluate the frailty of classifiers on new styles of known objects and unexpected instances of known classes. We also demonstrate two methods that improve surface variation robustness. Together our benchmarks may aid future work toward networks that learn fundamental class structure and also robustly generalize.
Code (2)
Tasks
Adversarial DefenseBenchmarkingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Benchmarking the Robustness of UAV Tracking Against Common Corruptions
The robustness of unmanned aerial vehicle (UAV) tracking is crucial in many tasks like surveillance and robotics. Despite its importance, little attention is paid to the performance of UAV trackers under common corruptio…
BenchmarkingOODFace: Benchmarking Robustness of Face Recognition under Common Corruptions and Appearance Variations
With the rise of deep learning, facial recognition technology has seen extensive research and rapid development. Although facial recognition is considered a mature technology, we find that existing open-source models and…
BenchmarkingFace RecognitionBenchmarking Robustness of 3D Object Detection to Common Corruptions
3D object detection is an important task in autonomous driving to perceive the surroundings. Despite the excellent performance, the existing 3D detectors lack the robustness to real-world corruptions caused by advers…
3D Object DetectionAutonomous DrivingBenchmarkingobject-detection+13D Common Corruptions and Data Augmentation
We introduce a set of image transformations that can be used as corruptions to evaluate the robustness of models as well as data augmentation mechanisms for training neural networks. The primary distinction of the propos…
BenchmarkingData AugmentationBenchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving
3D object detection is an important task in autonomous driving to perceive the surroundings. Despite the excellent performance, the existing 3D detectors lack the robustness to real-world corruptions caused by adverse we…
3D Object DetectionAutonomous DrivingBenchmarkingObject+3