Self Pre-training with Masked Autoencoders for Medical Image Classification and Segmentation
Masked Autoencoder (MAE) has recently been shown to be effective in pre-training Vision Transformers (ViT) for natural image analysis. By reconstructing full images from partially masked inputs, a ViT encoder aggregates contextual information to infer masked image regions. We believe that this context aggregation ability is particularly essential to the medical image domain where each anatomical structure is functionally and mechanically connected to other structures and regions. Because there is no ImageNet-scale medical image dataset for pre-training, we investigate a self pre-training paradigm with MAE for medical image analysis tasks. Our method pre-trains a ViT on the training set of the target data instead of another dataset. Thus, self pre-training can benefit more scenarios where pre-training data is hard to acquire. Our experimental results show that MAE self pre-training markedly improves diverse medical image tasks including chest X-ray disease classification, abdominal CT multi-organ segmentation, and MRI brain tumor segmentation. Code is available at https://github.com/cvlab-stonybrook/SelfMedMAE
Code (1)
Tasks
Brain Tumor Segmentationimage-classificationImage ClassificationLung Disease ClassificationMedical Image AnalysisMedical Image ClassificationOrgan SegmentationTumor SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Self Pre-training with Adaptive Mask Autoencoders for Variable-Contrast 3D Medical Imaging
The Masked Autoencoder (MAE) has recently demonstrated effectiveness in pre-training Vision Transformers (ViT) for analyzing natural images. By reconstructing complete images from partially masked inputs, the ViT encoder…
Self-supervised pseudo-colorizing of masked cells
Self-supervised learning, which is strikingly referred to as the dark matter of intelligence, is gaining more attention in biomedical applications of deep learning. In this work, we introduce a novel self-supervision obj…
Cell DetectionColorizationContrastive LearningSelf-Supervised Learning+1SelfMedHPM: Self Pre-training With Hard Patches Mining Masked Autoencoders For Medical Image Segmentation
In recent years, deep learning methods such as convolutional neural network (CNN) and transformers have made significant progress in CT multi-organ segmentation. However, CT multi-organ segmentation methods based on mask…
Image SegmentationMedical Image SegmentationOrgan SegmentationSegmentation+1Self Pre-training with Topology- and Spatiality-aware Masked Autoencoders for 3D Medical Image Segmentation
Masked Autoencoders (MAEs) have been shown to be effective in pre-training Vision Transformers (ViTs) for natural and medical image analysis problems. By reconstructing missing pixel/voxel information in visible patches,…
Image SegmentationMedical Image AnalysisMedical Image SegmentationSegmentation+1Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training
Medical vision-and-language pre-training provides a feasible solution to extract effective vision-and-language representations from medical images and texts. However, few studies have been dedicated to this field to faci…
Self-Supervised Learning