Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives

2023-08-03 · CVPR 2023 1 · Chuntao Ding, Zhichao Lu, Shangguang Wang, Ran Cheng, Vishnu Naresh Boddeti

Multi-task learning (MTL) seeks to learn a single model to accomplish multiple tasks by leveraging shared information among the tasks. Existing MTL models, however, have been known to suffer from negative interference among tasks. Efforts to mitigate task interference have focused on either loss/gradient balancing or implicit parameter partitioning with partial overlaps among the tasks. In this paper, we propose ETR-NLP to mitigate task interference through a synergistic combination of non-learnable primitives (NLPs) and explicit task routing (ETR). Our key idea is to employ non-learnable primitives to extract a diverse set of task-agnostic features and recombine them into a shared branch common to all tasks and explicit task-specific branches reserved for each task. The non-learnable primitives and the explicit decoupling of learnable parameters into shared and task-specific ones afford the flexibility needed for minimizing task interference. We evaluate the efficacy of ETR-NLP networks for both image-level classification and pixel-level dense prediction MTL problems. Experimental results indicate that ETR-NLP significantly outperforms state-of-the-art baselines with fewer learnable parameters and similar FLOPs across all datasets. Code is available at this \href{https://github.com/zhichao-lu/etr-nlp-mtl}.

📄 PDF Abstract BibTeX arXiv:2308.02066

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-Task Learning

FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts

2025-10-09 · Heming Zou, Yunliang Zang, Wutong Xu, Yao Zhu 외 arxiv

Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning method for foundation models, but it suffers from parameter interference, resulting in suboptimal performance. Although Mixture-of-Experts (MoE)…

parameter-efficient fine-tuning Computational Efficiency Mathematical Reasoning Question Answering

Crab$^{+}$: A Scalable and Unified Audio-Visual Scene Understanding Model with Explicit Cooperation

2026-03-04 · Dongnuan Cai, Henghui Du, Chang Zhou, Xi Chen 외 arxiv

Developing Audio-Visual Large Language Models (AV-LLMs) for unified scene understanding is pivotal in multimodal intelligence. While instruction tuning enables pre-trained models with multi-task abilities, we observe tha…

Multi-Task Learning Scene Understanding

Post-Hoc Merging is Not Enough: Many-Shot Model Merging with Loss-Gap Balancing

2026-06-15 · Kyungjin Im, Miru Kim, Chanin Eom, Minhae Kwon arxiv

Model merging has become a practical post-training strategy for building a single multi-task large language model (LLM) by combining multiple task-specialized models. However, most existing approaches rely on post-hoc me…

Toward Multi-class Anomaly Detection: Exploring Class-aware Unified Model against Inter-class Interference

2024-03-21 · Xi Jiang, Ying Chen, Qiang Nie, Jianlin Liu 외

In the context of high usability in single-class anomaly detection models, recent academic research has become concerned about the more complex multi-class anomaly detection. Although several papers have designed unified…

Anomaly Detection Decoder Multi-class Anomaly Detection

Little By Little: Continual Learning via Self-Activated Sparse Mixture-of-Rank Adaptive Learning

2025-06-26 · Haodong Lu, Chongyang Zhao, Jason Xue, Lina Yao 외

Continual learning (CL) with large pre-trained models is challenged by catastrophic forgetting and task interference. Existing LoRA-based Mixture-of-Experts (MoE) approaches mitigate forgetting by assigning and freezing …

Continual Learning Mixture-of-Experts