MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysis
The Vision Transformer (ViT) has demonstrated remarkable performance in Self-Supervised Learning (SSL) for 3D medical image analysis. Masked AutoEncoder (MAE) for feature pre-training can further unleash the potential of ViT on various medical vision tasks. However, due to large spatial sizes with much higher dimensions of 3D medical images, the lack of hierarchical design for MAE may hinder the performance of downstream tasks. In this paper, we propose a novel \textit{Mask in Mask (MiM)} pre-training framework for 3D medical images, which aims to advance MAE by learning discriminative representation from hierarchical visual tokens across varying scales. We introduce multiple levels of granularity for masked inputs from the volume, which are then reconstructed simultaneously ranging at both fine and coarse levels. Additionally, a cross-level alignment mechanism is applied to adjacent level volumes to enforce anatomical similarity hierarchically. Furthermore, we adopt a hybrid backbone to enhance the hierarchical representation learning efficiently during the pre-training. MiM was pre-trained on a large scale of available 3D volumetric images, \textit{i.e.,} Computed Tomography (CT) images containing various body parts. Extensive experiments on thirteen public datasets demonstrate the superiority of MiM over other SSL methods in organ/lesion/tumor segmentation and disease classification. We further scale up the MiM to large pre-training datasets with more than 10k volumes, showing that large-scale pre-training can further enhance the performance of downstream tasks. The improvement also concluded that the research community should pay more attention to the scale of the pre-training dataset towards the healthcare foundation model for 3D medical images.
Code (0)
등록된 구현이 없습니다.
Tasks
Computed Tomography (CT)Medical Image AnalysisRepresentation LearningSelf-Supervised LearningTumor SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Mask What Matters: Controllable Text-Guided Masking for Self-Supervised Medical Image Analysis
The scarcity of annotated data in specialized domains such as medical imaging presents significant challenges to training robust vision models. While self-supervised masked image modeling (MIM) offers a promising solutio…
Self-Supervised LearningRepresentation LearningMPS-AMS: Masked Patches Selection and Adaptive Masking Strategy Based Self-Supervised Medical Image Segmentation
Existing self-supervised learning methods based on contrastive learning and masked image modeling have demonstrated impressive performances. However, current masked image modeling methods are mainly utilized in natural i…
Contrastive LearningImage SegmentationMedical Image SegmentationSegmentation+2Self-supervised pseudo-colorizing of masked cells
Self-supervised learning, which is strikingly referred to as the dark matter of intelligence, is gaining more attention in biomedical applications of deep learning. In this work, we introduce a novel self-supervision obj…
Cell DetectionColorizationContrastive LearningSelf-Supervised Learning+1AnatoMask: Enhancing Medical Image Segmentation with Reconstruction-guided Self-masking
Due to the scarcity of labeled data, self-supervised learning (SSL) has gained much attention in 3D medical image segmentation, by extracting semantic representations from unlabeled data. Among SSL strategies, Masked ima…
Image SegmentationMedical Image SegmentationSelf-Supervised LearningSemantic SegmentationLearning Generalizable 3D Medical Image Representations from Mask-Guided Self-Supervision
Foundation models have transformed vision and language by learning general-purpose representations from large-scale unlabeled data, yet 3D medical imaging lacks analogous approaches. Existing self-supervised methods rely…
Self-Supervised Learning