Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contexts
Preserving maximal information is one of principles of designing self-supervised learning methodologies. To reach this goal, contrastive learning adopts an implicit way which is contrasting image pairs. However, we believe it is not fully optimal to simply use the contrastive estimation for preservation. Moreover, it is necessary and complemental to introduce an explicit solution to preserve more information. From this perspective, we introduce Preservational Learning to reconstruct diverse image contexts in order to preserve more information in learned representations. Together with the contrastive loss, we present Preservational Contrastive Representation Learning (PCRL) for learning self-supervised medical representations. PCRL provides very competitive results under the pretraining-finetuning protocol, outperforming both self-supervised and supervised counterparts in 5 classification/segmentation tasks substantially.
Code (2)
Tasks
Contrastive LearningRepresentation LearningSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Big Self-Supervised Models Advance Medical Image Classification
Self-supervised pretraining followed by supervised fine-tuning has seen success in image recognition, especially when labeled examples are scarce, but has received limited attention in medical image analysis. This paper …
ClassificationContrastive LearningGeneral Classificationimage-classification+5Localized Region Contrast for Enhancing Self-Supervised Learning in Medical Image Segmentation
Recent advancements in self-supervised learning have demonstrated that effective visual representations can be learned from unlabeled images. This has led to increased interest in applying self-supervised learning to the…
Contrastive LearningImage SegmentationMedical Image SegmentationOrgan Segmentation+3Efficient Self-Supervised Adaptation for Medical Image Analysis
Self-supervised adaptation (SSA) improves foundation model transfer to medical domains but is computationally prohibitive. Although parameter efficient fine-tuning methods such as LoRA have been explored for supervised a…
GPUMedical Image Analysisparameter-efficient fine-tuningSelf-supervised Model Based on Masked Autoencoders Advance CT Scans Classification
The coronavirus pandemic has been going on since the year 2019, and the trend is still not abating. Therefore, it is particularly important to classify medical CT scans to assist in medical diagnosis. At present, Supervi…
Medical DiagnosisSelf-Supervised LearningTransfer LearningLVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching
Obtaining large pre-trained models that can be fine-tuned to new tasks with limited annotated samples has remained an open challenge for medical imaging data. While pre-trained deep networks on ImageNet and vision-langua…
Brain Tumor ClassificationContrastive LearningDiabetic Retinopathy GradingGraph Matching+2