Are Natural Domain Foundation Models Useful for Medical Image Classification?
The deep learning field is converging towards the use of general foundation models that can be easily adapted for diverse tasks. While this paradigm shift has become common practice within the field of natural language processing, progress has been slower in computer vision. In this paper we attempt to address this issue by investigating the transferability of various state-of-the-art foundation models to medical image classification tasks. Specifically, we evaluate the performance of five foundation models, namely SAM, SEEM, DINOv2, BLIP, and OpenCLIP across four well-established medical imaging datasets. We explore different training settings to fully harness the potential of these models. Our study shows mixed results. DINOv2 consistently outperforms the standard practice of ImageNet pretraining. However, other foundation models failed to consistently beat this established baseline indicating limitations in their transferability to medical image classification tasks.
Code (1)
Tasks
image-classificationImage ClassificationMedical Image ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Biomedical SAM 2: Segment Anything in Biomedical Images and Videos
Medical image segmentation and video object segmentation are essential for diagnosing and analyzing diseases by identifying and measuring biological structures. Recent advances in natural domain have been driven by found…
Image SegmentationMedical Image SegmentationSegmentationSemantic Segmentation+3MedDINOv3: How to adapt vision foundation models for medical image segmentation?
Accurate segmentation of organs and tumors in CT and MRI scans is essential for diagnosis, treatment planning, and disease monitoring. While deep learning has advanced automated segmentation, most models remain task-spec…
Medical Image SegmentationAdapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains
Multi-modal foundation models are typically trained on millions of pairs of natural images and text captions, frequently obtained through web-crawling approaches. Although such models depict excellent generative capabili…
Advancements in Medical Image Classification through Fine-Tuning Natural Domain Foundation Models
Using massive datasets, foundation models are large-scale, pre-trained models that perform a wide range of tasks. These models have shown consistently improved results with the introduction of new methods. It is crucial …
Classificationimage-classificationImage ClassificationMedical Image Classification+1Does DINOv3 Set a New Medical Vision Standard? Benchmarking 2D and 3D Classification, Segmentation, and Registration
The advent of large-scale vision foundation models, pre-trained on diverse natural images, has marked a paradigm shift in computer vision. However, how the frontier vision foundation models' efficacies transfer to specia…
3D Reconstruction3D Classification