Efficient Self-Supervised Adaptation for Medical Image Analysis
Self-supervised adaptation (SSA) improves foundation model transfer to medical domains but is computationally prohibitive. Although parameter efficient fine-tuning methods such as LoRA have been explored for supervised adaptation, their effectiveness for SSA remains unknown. In this work, we introduce efficient self-supervised adaptation (ESSA), a framework that applies parameter-efficient fine-tuning techniques to SSA with the aim of reducing computational cost and improving adaptation performance. Among the methods tested, Attention Projection Layer Adaptation (APLA) sets a new state-of-the-art, consistently surpassing full-parameter SSA and supervised fine-tuning across diverse medical tasks, while reducing GPU memory by up to 40.1% and increasing training throughput by 25.2%, all while maintaining inference efficiency.
Code (1)
Tasks
GPUMedical Image Analysisparameter-efficient fine-tuningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Review of Predictive and Contrastive Self-supervised Learning for Medical Images
Over the last decade, supervised deep learning on manually annotated big data has been progressing significantly on computer vision tasks. But the application of deep learning in medical image analysis was limited by the…
Deep LearningMedical Image AnalysisSelf-Supervised LearningMultitask Multimodal Self-Supervised Learning for Medical Images
This thesis works to address a pivotal challenge in medical image analysis: the reliance on extensive labeled datasets, which are often limited due to the need for expert annotation and constrained by privacy and legal i…
Self-Supervised LearningDomain AdaptationDINO-CXR: A self supervised method based on vision transformer for chest X-ray classification
The limited availability of labeled chest X-ray datasets is a significant bottleneck in the development of medical imaging methods. Self-supervised learning (SSL) can mitigate this problem by training models on unlabeled…
COVID-19 DiagnosisImage ClassificationMedical Image AnalysisMedical Image Classification+4Domain Adaptation for Medical Image Analysis: A Survey
Machine learning techniques used in computer-aided medical image analysis usually suffer from the domain shift problem caused by different distributions between source/reference data and target data. As a promising solut…
Domain AdaptationMedical Image AnalysisSurveyCOSMOS: Cross-Modality Unsupervised Domain Adaptation for 3D Medical Image Segmentation based on Target-aware Domain Translation and Iterative Self-Training
Recent advances in deep learning-based medical image segmentation studies achieve nearly human-level performance when in fully supervised condition. However, acquiring pixel-level expert annotations is extremely expensiv…
Domain AdaptationImage SegmentationMedical Image SegmentationSegmentation+2