Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
Self-supervised representation learning approaches have recently surpassed their supervised learning counterparts on downstream tasks like object detection and image classification. Somewhat mysteriously the recent gains in performance come from training instance classification models, treating each image and it's augmented versions as samples of a single class. In this work, we first present quantitative experiments to demystify these gains. We demonstrate that approaches like MOCO and PIRL learn occlusion-invariant representations. However, they fail to capture viewpoint and category instance invariance which are crucial components for object recognition. Second, we demonstrate that these approaches obtain further gains from access to a clean object-centric training dataset like Imagenet. Finally, we propose an approach to leverage unstructured videos to learn representations that possess higher viewpoint invariance. Our results show that the learned representations outperform MOCOv2 trained on the same data in terms of invariances encoded and the performance on downstream image classification and semantic segmentation tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral Classificationimage-classificationImage ClassificationObjectobject-detectionObject DetectionObject RecognitionRepresentation LearningSelf-Supervised LearningSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Theoretical Characterization of Optimal Data Augmentations in Self-Supervised Learning
Data augmentations play an important role in the recent success of Self-Supervised Learning (SSL). While commonly viewed as encoding invariances into the learned representations, this interpretation overlooks the impact …
Self-Supervised LearningAmortised Invariance Learning for Contrastive Self-Supervision
Contrastive self-supervised learning methods famously produce high quality transferable representations by learning invariances to different data augmentations. Invariances established during pre-training can be interpre…
Contrastive LearningRepresentation LearningSelf-Supervised LearningDomain Confused Contrastive Learning for Unsupervised Domain Adaptation
In this work, we study Unsupervised Domain Adaptation (UDA) in a challenging self-supervised approach. One of the difficulties is how to learn task discrimination in the absence of target labels. Unlike previous literatu…
Contrastive LearningDomain AdaptationUnsupervised Domain AdaptationSkeleton-Contrastive 3D Action Representation Learning
This paper strives for self-supervised learning of a feature space suitable for skeleton-based action recognition. Our proposal is built upon learning invariances to input skeleton representations and various skeleton au…
Action RecognitionContrastive LearningFew-Shot Skeleton-Based Action RecognitionRepresentation Learning+4Automatic Data Augmentation Selection and Parametrization in Contrastive Self-Supervised Speech Representation Learning
Contrastive learning enables learning useful audio and speech representations without ground-truth labels by maximizing the similarity between latent representations of similar signal segments. In this framework various …
Contrastive LearningData AugmentationRepresentation LearningSpeech Representation Learning