On Evolving Attention Towards Domain Adaptation
Towards better unsupervised domain adaptation (UDA). Recently, researchers propose various domain-conditioned attention modules and make promising progresses. However, considering that the configuration of attention, i.e., the type and the position of attention module, affects the performance significantly, it is more generalized to optimize the attention configuration automatically to be specialized for arbitrary UDA scenario. For the first time, this paper proposes EvoADA: a novel framework to evolve the attention configuration for a given UDA task without human intervention. In particular, we propose a novel search space containing diverse attention configurations. Then, to evaluate the attention configurations and make search procedure UDA-oriented (transferability + discrimination), we apply a simple and effective evaluation strategy: 1) training the network weights on two domains with off-the-shelf domain adaptation methods; 2) evolving the attention configurations under the guide of the discriminative ability on the target domain. Experiments on various kinds of cross-domain benchmarks, i.e., Office-31, Office-Home, CUB-Paintings, and Duke-Market-1510, reveal that the proposed EvoADA consistently boosts multiple state-of-the-art domain adaptation approaches, and the optimal attention configurations help them achieve better performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationPartial Domain AdaptationUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
Progressive Conservative Adaptation for Evolving Target Domains
Conventional domain adaptation typically transfers knowledge from a source domain to a stationary target domain. However, in many real-world cases, target data usually emerge sequentially and have continuously evolving d…
Domain AdaptationMeta-LearningRotated MNISTLearning to Adapt to Evolving Domains
Domain adaptation aims at knowledge transfer from a labeled source domain to an unlabeled target domain. Current domain adaptation methods have made substantial advances in adapting discrete domains. However, this can be…
Domain AdaptationMeta-LearningTransfer LearningUnsupervised Domain AdaptationEvolving Domain Adaptation of Pretrained Language Models for Text Classification
Adapting pre-trained language models (PLMs) for time-series text classification amidst evolving domain shifts (EDS) is critical for maintaining accuracy in applications like stance detection. This study benchmarks the ef…
Domain AdaptationStance Detectiontext-classificationText Classification+1Continuous Manifold Based Adaptation for Evolving Visual Domains
We pose the following question: what happens when test data not only differs from training data, but differs from it in a continually evolving way? The classic domain adaptation paradigm considers the world to be separat…
Domain AdaptationGradual Domain Adaptation via Self-Training of Auxiliary Models
Domain adaptation becomes more challenging with increasing gaps between source and target domains. Motivated from an empirical analysis on the reliability of labeled source data for the use of distancing target domains, …
Domain AdaptationSemi-supervised Domain Adaptation