Agile Multi-Source-Free Domain Adaptation
Efficiently utilizing rich knowledge in pretrained models has become a critical topic in the era of large models. This work focuses on adaptively utilizing knowledge from multiple source-pretrained models to an unlabeled target domain without accessing the source data. Despite being a practically useful setting, existing methods require extensive parameter tuning over each source model, which is computationally expensive when facing abundant source domains or larger source models. To address this challenge, we propose a novel approach which is free of the parameter tuning over source backbones. Our technical contribution lies in the Bi-level ATtention ENsemble (Bi-ATEN) module, which learns both intra-domain weights and inter-domain ensemble weights to achieve a fine balance between instance specificity and domain consistency. By slightly tuning source bottlenecks, we achieve comparable or even superior performance on a challenging benchmark DomainNet with less than 3% trained parameters and 8 times of throughput compared with SOTA method. Furthermore, with minor modifications, the proposed module can be easily equipped to existing methods and gain more than 4% performance boost. Code is available at https://github.com/TL-UESTC/Bi-ATEN.
Code (1)
Tasks
Domain AdaptationSource-Free Domain AdaptationSpecificitySimilar Papers 제목 키워드 기반
Uncertainty-Guided Mixup for Semi-Supervised Domain Adaptation without Source Data
Present domain adaptation methods usually perform explicit representation alignment by simultaneously accessing the source data and target data. However, the source data are not always available due to the privacy preser…
Domain AdaptationPrivacy PreservingSemi-supervised Domain AdaptationSource-Free Domain AdaptationAgile Domain Adaptation
Domain adaptation investigates the problem of leveraging knowledge from a well-labeled source domain to an unlabeled target domain, where the two domains are drawn from different data distributions. Because of the distri…
Domain AdaptationSemi-Supervised Domain Adaptation with Auto-Encoder via Simultaneous Learning
We present a new semi-supervised domain adaptation framework that combines a novel auto-encoder-based domain adaptation model with a simultaneous learning scheme providing stable improvements over state-of-the-art domain…
Domain AdaptationSemi-supervised Domain AdaptationCoNMix for Source-free Single and Multi-target Domain Adaptation
This work introduces the novel task of Source-free Multi-target Domain Adaptation and proposes adaptation framework comprising of \textbf{Co}nsistency with \textbf{N}uclear-Norm Maximization and \textbf{Mix}Up knowledge …
Domain AdaptationKnowledge DistillationMulti-target Domain AdaptationPseudo LabelTemporal-Spectral Alignment with Frequency Adaptation for Source-Free Time-Series Adaptation
The goal of source-free domain adaptation (SFDA) for time-series data is to transfer knowledge from a pre-trained source model to an unlabeled target domain without requiring access to source data, while addressing featu…
Source-Free Domain Adaptation