paper-with-me

홈 › Papers

Transfer or Self-Supervised? Bridging the Performance Gap in Medical Imaging

2024-07-08 · Zehui Zhao, Laith Alzubaidi, Jinglan Zhang, Ye Duan, Usman Naseem, Yuantong Gu

Recently, transfer learning and self-supervised learning have gained significant attention within the medical field due to their ability to mitigate the challenges posed by limited data availability, improve model generalisation, and reduce computational expenses. Transfer learning and self-supervised learning hold immense potential for advancing medical research. However, it is crucial to recognise that transfer learning and self-supervised learning architectures exhibit distinct advantages and limitations, manifesting variations in accuracy, training speed, and robustness. This paper compares the performance and robustness of transfer learning and self-supervised learning in the medical field. Specifically, we pre-trained two models using the same source domain datasets with different pre-training methods and evaluated them on small-sized medical datasets to identify the factors influencing their final performance. We tested data with several common issues in medical domains, such as data imbalance, data scarcity, and domain mismatch, through comparison experiments to understand their impact on specific pre-trained models. Finally, we provide recommendations to help users apply transfer learning and self-supervised learning methods in medical areas, and build more convenient and efficient deployment strategies.

📄 PDF Abstract BibTeX arXiv:2407.05592

Code (0)

등록된 구현이 없습니다.

Tasks

Self-Supervised LearningTransfer Learning

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음

Similar Papers 제목 키워드 기반

A Systematic Benchmarking Analysis of Transfer Learning for Medical Image Analysis

2021-08-12 · Mohammad Reza Hosseinzadeh Taher, Fatemeh Haghighi, Ruibin Feng, Michael B. Gotway 외

Transfer learning from supervised ImageNet models has been frequently used in medical image analysis. Yet, no large-scale evaluation has been conducted to benchmark the efficacy of newly-developed pre-training techniques…

BenchmarkingMedical Image AnalysisTransfer Learning

How Transferable Are Self-supervised Features in Medical Image Classification Tasks?

2021-08-23 · Tuan Truong, Sadegh Mohammadi, Matthias Lenga

Transfer learning has become a standard practice to mitigate the lack of labeled data in medical classification tasks. Whereas finetuning a downstream task using supervised ImageNet pretrained features is straightforward…

Classificationimage-classificationImage ClassificationLinear evaluation+3

Efficient Self-Supervised Adaptation for Medical Image Analysis

2025-03-24 · Moein Sorkhei, Emir Konuk, Jingyu Guo, Christos Matsoukas 외

Self-supervised adaptation (SSA) improves foundation model transfer to medical domains but is computationally prohibitive. Although parameter efficient fine-tuning methods such as LoRA have been explored for supervised a…

GPUMedical Image Analysisparameter-efficient fine-tuning

ACE: Anatomically Consistent Embeddings in Composition and Decomposition

2025-01-17 · Ziyu Zhou, Haozhe Luo, Mohammad Reza Hosseinzadeh Taher, Jiaxuan Pang 외

Medical images acquired from standardized protocols show consistent macroscopic or microscopic anatomical structures, and these structures consist of composable/decomposable organs and tissues, but existing self-supervis…

Few-Shot LearningImage CroppingSelf-Supervised Learning

Self-supervised contrastive learning of echocardiogram videos enables label-efficient cardiac disease diagnosis

2022-07-23 · Gregory Holste, Evangelos K. Oikonomou, Bobak J. Mortazavi, Zhangyang Wang 외

Advances in self-supervised learning (SSL) have shown that self-supervised pretraining on medical imaging data can provide a strong initialization for downstream supervised classification and segmentation. Given the diff…

ClassificationContrastive LearningRepresentation LearningSelf-Supervised Learning+1