Learning Background Invariance Improves Generalization and Robustness in Self-Supervised Learning on ImageNet and Beyond
Recent progress in self-supervised learning has demonstrated promising results in multiple visual tasks. An important ingredient in high-performing self-supervised methods is the use of data augmentation by training models to place different augmented views of the same image nearby in embedding space. However, commonly used augmentation pipelines treat images holistically, ignoring the semantic relevance of parts of an image—e.g. a subject vs. a background—which can lead to the learning of spurious correlations. Our work addresses this problem by investigating a class of simple, yet highly effective “background augmentations", which encourage models to focus on semantically-relevant content by discouraging them from focusing on image backgrounds. Through a systematic, comprehensive investigation, we show that background augmentations lead to improved generalization with substantial improvements ($\sim$1-2% on ImageNet) in performance across a spectrum of state-of-the-art self-supervised methods (MoCo-v2, BYOL, SwAV) on a variety of tasks, even enabling performance on par with the supervised baseline. We also find improved label efficiency with even larger performance improvements in limited-labels settings (up to 4.2%). Further, we find improved training efficiency, attaining a benchmark accuracy of 74.4%, outperforming many recent self-supervised learning methods trained for 800-1000 epochs, in only 100 epochs. Importantly, we also demonstrate that background augmentations boost generalization and robustness to a number of out-of-distribution settings, including ImageNet-9, natural adversarial examples, adversarial attacks, ImageNet-Renditions and ImageNet ReaL. We also make progress in completely unsupervised saliency detection, in the process of generating saliency masks that we use for background augmentations.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationSaliency DetectionSelf-Supervised LearningUnsupervised Saliency DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Shift Invariance Can Reduce Adversarial Robustness
Shift invariance is a critical property of CNNs that improves performance on classification. However, we show that invariance to circular shifts can also lead to greater sensitivity to adversarial attacks. We first chara…
Adversarial RobustnessInvariance vs Robustness of Neural Networks
Neural networks achieve human-level accuracy on many standard datasets used in image classification. The next step is to achieve better generalization to natural (or non-adversarial) perturbations as well as known pixel-…
Adversarial Robustnessimage-classificationImage ClassificationBackground Invariance Testing According to Semantic Proximity
In many applications, machine learned (ML) models are required to hold some invariance qualities, such as rotation, size, intensity, and background invariance. Unlike many types of variance, the variants of background sc…
Object RecognitionJoint Self-Supervised Learning for Vision-based Reinforcement Learning
Vision-based reinforcement learning requires efficient and robust representations of image-based observations, especially when the images contain distracting (task-irrelevant) elements such as shadows, clouds, and light.…
Autonomous Drivingcontinuous-controlContinuous Controlreinforcement-learning+3CoDo: Contrastive Learning with Downstream Background Invariance for Detection
The prior self-supervised learning researches mainly select image-level instance discrimination as pretext task. It achieves a fantastic classification performance that is comparable to supervised learning methods. Howev…
Contrastive LearningData AugmentationObjectobject-detection+4