Can Self-Supervised Representation Learning Methods Withstand Distribution Shifts and Corruptions?
Self-supervised learning in computer vision aims to leverage the inherent structure and relationships within data to learn meaningful representations without explicit human annotation, enabling a holistic understanding of visual scenes. Robustness in vision machine learning ensures reliable and consistent performance, enhancing generalization, adaptability, and resistance to noise, variations, and adversarial attacks. Self-supervised paradigms, namely contrastive learning, knowledge distillation, mutual information maximization, and clustering, have been considered to have shown advances in invariant learning representations. This work investigates the robustness of learned representations of self-supervised learning approaches focusing on distribution shifts and image corruptions in computer vision. Detailed experiments have been conducted to study the robustness of self-supervised learning methods on distribution shifts and image corruptions. The empirical analysis demonstrates a clear relationship between the performance of learned representations within self-supervised paradigms and the severity of distribution shifts and corruptions. Notably, higher levels of shifts and corruptions are found to significantly diminish the robustness of the learned representations. These findings highlight the critical impact of distribution shifts and image corruptions on the performance and resilience of self-supervised learning methods, emphasizing the need for effective strategies to mitigate their adverse effects. The study strongly advocates for future research in the field of self-supervised representation learning to prioritize the key aspects of safety and robustness in order to ensure practical applicability. The source code and results are available on GitHub.
Code (1)
Tasks
Contrastive LearningKnowledge DistillationRepresentation LearningSelf-Supervised LearningSimilar Papers 제목 키워드 기반
Self-Supervised Graph Transformer for Deepfake Detection
Deepfake detection methods have shown promising results in recognizing forgeries within a given dataset, where training and testing take place on the in-distribution dataset. However, their performance deteriorates signi…
Contrastive LearningDeepFake DetectionFace SwappingVideo CompressionImproving out-of-distribution generalization via multi-task self-supervised pretraining
Self-supervised feature representations have been shown to be useful for supervised classification, few-shot learning, and adversarial robustness. We show that features obtained using self-supervised learning are compara…
Adversarial RobustnessDomain GeneralizationFew-Shot LearningMulti-Task Learning+2Learning Robust Reasoning through Guided Adversarial Self-Play
Reinforcement learning from verifiable rewards (RLVR) produces strong reasoning models, yet they can fail catastrophically when the conditioning context is fallible (e.g., corrupted chain-of-thought, misleading partial s…
Reinforcement LearningTempo vs. Pitch: understanding self-supervised tempo estimation
Self-supervision methods learn representations by solving pretext tasks that do not require human-generated labels, alleviating the need for time-consuming annotations. These methods have been applied in computer vision,…
Information RetrievalMusic Information RetrievalRetrievalA Probabilistic Model Behind Self-Supervised Learning
In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels. A common task is to classify augmentations or different modalities of the data, which share semantic content …
modelRepresentation LearningSelf-Supervised Learning