A Broad Study of Pre-training for Domain Generalization and Adaptation
Deep models must learn robust and transferable representations in order to perform well on new domains. While domain transfer methods (e.g., domain adaptation, domain generalization) have been proposed to learn transferable representations across domains, they are typically applied to ResNet backbones pre-trained on ImageNet. Thus, existing works pay little attention to the effects of pre-training on domain transfer tasks. In this paper, we provide a broad study and in-depth analysis of pre-training for domain adaptation and generalization, namely: network architectures, size, pre-training loss, and datasets. We observe that simply using a state-of-the-art backbone outperforms existing state-of-the-art domain adaptation baselines and set new baselines on Office-Home and DomainNet improving by 10.7\% and 5.5\%. We hope that this work can provide more insights for future domain transfer research.
Code (1)
Tasks
Domain AdaptationDomain GeneralizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Semantic-aware Message Broadcasting for Efficient Unsupervised Domain Adaptation
Vision transformer has demonstrated great potential in abundant vision tasks. However, it also inevitably suffers from poor generalization capability when the distribution shift occurs in testing (i.e., out-of-distributi…
Domain AdaptationPseudo LabelUnsupervised Domain AdaptationA Survey on Deep Reinforcement Learning-based Approaches for Adaptation and Generalization
Deep Reinforcement Learning (DRL) aims to create intelligent agents that can learn to solve complex problems efficiently in a real-world environment. Typically, two learning goals: adaptation and generalization are used …
Deep Reinforcement Learningreinforcement-learningReinforcement Learning (RL)An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift
Preference tuning aligns base language models to human judgments of quality, helpfulness, or safety by optimizing over explicit preference signals rather than likelihood alone. Prior work has shown that preference tuning…
Role of Image Acquisition and Patient Phenotype Variations in Automatic Segmentation Model Generalization
Purpose: This study evaluated the out-of-domain performance and generalization capabilities of automated medical image segmentation models, with a particular focus on adaptation to new image acquisitions and disease type…
Image SegmentationMedical Image SegmentationSemantic SegmentationSAM-DA: Decoder Adapter for Efficient Medical Domain Adaptation
This paper addresses the domain adaptation challenge for semantic segmentation in medical imaging. Despite the impressive performance of recent foundational segmentation models like SAM on natural images, they struggle w…
DecoderDomain AdaptationSegmentationSemantic Segmentation