Dive into Self-Supervised Learning for Medical Image Analysis: Data, Models and Tasks
Self-supervised learning (SSL) has achieved remarkable performance in various medical imaging tasks by dint of priors from massive unlabelled data. However, regarding a specific downstream task, there is still a lack of an instruction book on how to select suitable pretext tasks and implementation details throughout the standard ``pretrain-then-finetune'' workflow. In this work, we focus on exploiting the capacity of SSL in terms of four realistic and significant issues: (1) the impact of SSL on imbalanced datasets, (2) the network architecture, (3) the applicability of upstream tasks to downstream tasks and (4) the stacking effect of SSL and common policies for deep learning. We provide a large-scale, in-depth and fine-grained study through extensive experiments on predictive, contrastive, generative and multi-SSL algorithms. Based on the results, we have uncovered several insights. Positively, SSL advances class-imbalanced learning mainly by boosting the performance of the rare class, which is of interest to clinical diagnosis. Unfortunately, SSL offers marginal or even negative returns in some cases, including severely imbalanced and relatively balanced data regimes, as well as combinations with common training policies. Our intriguing findings provide practical guidelines for the usage of SSL in the medical context and highlight the need for developing universal pretext tasks to accommodate diverse application scenarios.
Code (2)
Tasks
Medical Image AnalysisSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exploring Intrinsic Properties of Medical Images for Self-Supervised Binary Semantic Segmentation
Recent advancements in self-supervised learning have unlocked the potential to harness unlabeled data for auxiliary tasks, facilitating the learning of beneficial priors. This has been particularly advantageous in fields…
DecoderImage SegmentationMedical Image AnalysisMedical Image Segmentation+3Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contexts
Preserving maximal information is one of principles of designing self-supervised learning methodologies. To reach this goal, contrastive learning adopts an implicit way which is contrasting image pairs. However, we belie…
Contrastive LearningRepresentation LearningSelf-Supervised LearningMAESIL: Masked Autoencoder for Enhanced Self-supervised Medical Image Learning
Training deep learning models for three-dimensional (3D) medical imaging, such as Computed Tomography (CT), is fundamentally challenged by the scarcity of labeled data. While pre-training on natural images is common, it …
Self-Supervised LearningComputational EfficiencyMedical visual question answering using joint self-supervised learning
Visual Question Answering (VQA) becomes one of the most active research problems in the medical imaging domain. A well-known VQA challenge is the intrinsic diversity between the image and text modalities, and in the medi…
DecoderDiversityMedical Visual Question AnsweringQuestion Answering+3A Systematic Benchmarking Analysis of Transfer Learning for Medical Image Analysis
Transfer learning from supervised ImageNet models has been frequently used in medical image analysis. Yet, no large-scale evaluation has been conducted to benchmark the efficacy of newly-developed pre-training techniques…
BenchmarkingMedical Image AnalysisTransfer Learning