M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework
Multi-domain recommendation and multi-task recommendation have demonstrated their effectiveness in leveraging common information from different domains and objectives for comprehensive user modeling. Nonetheless, the practical recommendation usually faces multiple domains and tasks simultaneously, which cannot be well-addressed by current methods. To this end, we introduce M3oE, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. M3oE integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives. We leverage three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences respectively to address the complex dependencies among multiple domains and tasks in a disentangled manner. Additionally, we design a two-level fusion mechanism for precise control over feature extraction and fusion across diverse domains and tasks. The framework's adaptability is further enhanced by applying AutoML technique, which allows dynamic structure optimization. To the best of the authors' knowledge, our M3oE is the first effort to solve multi-domain multi-task recommendation self-adaptively. Extensive experiments on two benchmark datasets against diverse baselines demonstrate M3oE's superior performance. The implementation code is available to ensure reproducibility.
Code (1)
Tasks
AutoMLMixture-of-ExpertsSimilar Papers 제목 키워드 기반
FRAME: Learning the Adaptation Domain with a Mixture of Fractional-Fourier Experts
Parameter-efficient fine-tuning (PEFT) reparameterizes weight updates in a fixed basis: low-rank adapters operate in the spatial domain, while a recent line of spectral methods operates in a fixed Fourier domain. We argu…
parameter-efficient fine-tuningMALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning
Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA have significantly improved the adaptation of LLMs to downstream tasks in a resource-efficient manner. However, in multi-task scenarios, challenges such as trainin…
Computational EfficiencyMixture-of-ExpertsMulti-Task Learningparameter-efficient fine-tuningSeeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
Multimodal Mixture-of-Experts (MoE) models have achieved remarkable performance on vision-language tasks. However, we identify a puzzling phenomenon termed Seeing but Not Thinking: models accurately perceive image conten…
Visual ReasoningSpeechMoE2: Mixture-of-Experts Model with Improved Routing
Mixture-of-experts based acoustic models with dynamic routing mechanisms have proved promising results for speech recognition. The design principle of router architecture is important for the large model capacity and hig…
Computational EfficiencyMixture-of-Expertsmodelspeech-recognition+1Multi-Source Domain Adaptation with Mixture of Experts
We propose a mixture-of-experts approach for unsupervised domain adaptation from multiple sources. The key idea is to explicitly capture the relationship between a target example and different source domains. This relati…
Domain AdaptationMixture-of-ExpertsPart-Of-Speech TaggingSentiment Analysis+1