Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
Self-supervision provides effective representations for downstream tasks without requiring labels. However, existing approaches lag behind fully supervised training and are often not thought beneficial beyond obviating or reducing the need for annotations. We find that self-supervision can benefit robustness in a variety of ways, including robustness to adversarial examples, label corruption, and common input corruptions. Additionally, self-supervision greatly benefits out-of-distribution detection on difficult, near-distribution outliers, so much so that it exceeds the performance of fully supervised methods. These results demonstrate the promise of self-supervision for improving robustness and uncertainty estimation and establish these tasks as new axes of evaluation for future self-supervised learning research.
Code (4)
Tasks
Anomaly DetectionOutlier DetectionOut-of-Distribution DetectionSelf-Supervised LearningUnsupervised Anomaly DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
STRUDEL: Self-Training with Uncertainty Dependent Label Refinement across Domains
We propose an unsupervised domain adaptation (UDA) approach for white matter hyperintensity (WMH) segmentation, which uses Self-Training with Uncertainty DEpendent Label refinement (STRUDEL). Self-training has recently b…
Domain AdaptationPseudo LabelSegmentationUnsupervised Domain AdaptationSelf-training with dual uncertainty for semi-supervised medical image segmentation
In the field of semi-supervised medical image segmentation, the shortage of labeled data is the fundamental problem. How to effectively learn image features from unlabeled images to improve segmentation accuracy is the m…
Image SegmentationMedical Image SegmentationSegmentationSemantic Segmentation+1Diversified Ensemble of Independent Sub-Networks for Robust Self-Supervised Representation Learning
Ensembling a neural network is a widely recognized approach to enhance model performance, estimate uncertainty, and improve robustness in deep supervised learning. However, deep ensembles often come with high computation…
DiversityEnsemble LearningOut-of-Distribution DetectionRepresentation LearningUncertainty-Supervised Interpretable and Robust Evidential Segmentation
Uncertainty estimation has been widely studied in medical image segmentation as a tool to provide reliability, particularly in deep learning approaches. However, previous methods generally lack effective supervision in u…
Medical Image SegmentationEnsemble Distribution Distillation for Self-Supervised Human Activity Recognition
Human Activity Recognition (HAR) has seen significant advancements with the adoption of deep learning techniques, yet challenges remain in terms of data requirements, reliability and robustness. This paper explores a nov…
Human Activity RecognitionSelf-Supervised LearningData Augmentation