Transferability Estimation for Semantic Segmentation Task
Transferability estimation is a fundamental problem in transfer learning to predict how good the performance is when transferring a source model (or source task) to a target task. With the guidance of transferability score, we can efficiently select the highly transferable source models without performing the real transfer in practice. Recent analytical transferability metrics are mainly designed for image classification problem, and currently there is no specific investigation for the transferability estimation of semantic segmentation task, which is an essential problem in autonomous driving, medical image analysis, etc. Consequently, we further extend the recent analytical transferability metric OTCE (Optimal Transport based Conditional Entropy) score to the semantic segmentation task. The challenge in applying the OTCE score is the high dimensional segmentation output, which is difficult to find the optimal coupling between so many pixels under an acceptable computation cost. Thus we propose to randomly sample N pixels for computing OTCE score and take the expectation over K repetitions as the final transferability score. Experimental evaluation on Cityscapes, BDD100K and GTA5 datasets demonstrates that the OTCE score highly correlates with the transfer performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous Drivingimage-classificationImage ClassificationMedical Image AnalysisSegmentationSemantic SegmentationTransfer LearningSimilar Papers 제목 키워드 기반
Transferability Estimation using Bhattacharyya Class Separability
Transfer learning has become a popular method for leveraging pre-trained models in computer vision. However, without performing computationally expensive fine-tuning, it is difficult to quantify which pre-trained source …
Classificationimage-classificationImage ClassificationSemantic Segmentation+1Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation
Transfer learning is a critical technique in training deep neural networks for the challenging medical image segmentation task that requires enormous resources. With the abundance of medical image data, many research ins…
Image SegmentationMedical Image SegmentationSegmentationSemantic Segmentation+1Topology-Driven Transferability Estimation of Medical Foundation Models for Segmentation
The advent of large-scale self-supervised learning (SSL) has produced a vast zoo of medical foundation models. However, selecting optimal medical foundation models for specific segmentation tasks remains a computational …
Self-Supervised LearningTopology-Driven Transferability Estimation for 3D Medical Vision Foundation Models
The growing number of medical vision foundation models highlights the need for effective model selection. However, mainstream selection methods rely on exhaustive fine-tuning, which is computationally expensive. Most of …
Unleashing the power of Neural Collapse for Transferability Estimation
Transferability estimation aims to provide heuristics for quantifying how suitable a pre-trained model is for a specific downstream task, without fine-tuning them all. Prior studies have revealed that well-trained models…
Fairnessimage-classificationImage ClassificationSemantic Segmentation+2