Multi-Source Domain Adaptation with Mixture of Experts
We propose a mixture-of-experts approach for unsupervised domain adaptation from multiple sources. The key idea is to explicitly capture the relationship between a target example and different source domains. This relationship, expressed by a point-to-set metric, determines how to combine predictors trained on various domains. The metric is learned in an unsupervised fashion using meta-training. Experimental results on sentiment analysis and part-of-speech tagging demonstrate that our approach consistently outperforms multiple baselines and can robustly handle negative transfer.
Code (1)
Tasks
Domain AdaptationMixture-of-ExpertsPart-Of-Speech TaggingSentiment AnalysisUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
Transformer Based Multi-Source Domain Adaptation
In practical machine learning settings, the data on which a model must make predictions often come from a different distribution than the data it was trained on. Here, we investigate the problem of unsupervised multi-sou…
Domain AdaptationMixture-of-ExpertsMALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning
Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA have significantly improved the adaptation of LLMs to downstream tasks in a resource-efficient manner. However, in multi-task scenarios, challenges such as trainin…
Computational EfficiencyMixture-of-ExpertsMulti-Task Learningparameter-efficient fine-tuningSTEM: An Approach to Multi-Source Domain Adaptation With Guarantees
Multi-source Domain Adaptation (MSDA) is more practical but challenging than the conventional unsupervised domain adaptation due to the involvement of diverse multiple data sources. Two fundamental challenges of MSDA are…
STEM: An Approach to Multi-Source Domain Adaptation With Guarantees
Multi-source Domain Adaptation (MSDA) is more practical but challenging than the conventional unsupervised domain adaptation due to the involvement of diverse multiple data sources. Two fundamental challenges of MSDA…
Domain AdaptationMulti-Source Unsupervised Domain AdaptationUnsupervised Domain AdaptationScalable Multi-Domain Adaptation of Language Models using Modular Experts
Domain-specific adaptation is critical to maximizing the performance of pre-trained language models (PLMs) on one or multiple targeted tasks, especially under resource-constrained use cases, such as edge devices. However…
Domain AdaptationGeneral KnowledgeMixture-of-Experts