CALM: Consensus-Aware Localized Merging for Multi-Task Learning
Model merging aims to integrate the strengths of multiple fine-tuned models into a unified model while preserving task-specific capabilities. Existing methods, represented by task arithmetic, are typically classified into global- and local-aware methods. However, global-aware methods inevitably cause parameter interference, while local-aware methods struggle to maintain the effectiveness of task-specific details in the merged model. To address these limitations, we propose a Consensus-Aware Localized Merging (CALM) method which incorporates localized information aligned with global task consensus, ensuring its effectiveness post-merging. CALM consists of three key components: (1) class-balanced entropy minimization sampling, providing a more flexible and reliable way to leverage unsupervised data; (2) an efficient-aware framework, selecting a small set of tasks for sequential merging with high scalability; (3) a consensus-aware mask optimization, aligning localized binary masks with global task consensus and merging them conflict-free. Experiments demonstrate the superiority and robustness of our CALM, significantly outperforming existing methods and achieving performance close to traditional MTL.
Code (1)
Tasks
Multi-Task LearningTask ArithmeticMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Localized Data-driven Consensus Control
This paper considers a localized data-driven consensus problem for leader-follower multi-agent systems with unknown discrete-time agent dynamics, where each follower computes its local control gain using only their local…
CALM-Net: Curvature-Aware LiDAR Point Cloud-based Multi-Branch Neural Network for Vehicle Re-Identification
This paper presents CALM-Net, a curvature-aware LiDAR point cloud-based multi-branch neural network for vehicle re-identification. The proposed model addresses the challenge of learning discriminative and complementary f…
Vehicle Re-IdentificationPoint CloudsCALM: Culturally Self-Aware Language Models
Cultural awareness in language models is the capacity to understand and adapt to diverse cultural contexts. However, most existing approaches treat culture as static background knowledge, overlooking its dynamic and evol…
Contrastive LearningLocalized Learning of Robust Controllers for Networked Systems with Dynamic Topology
This paper addresses the problem of controller synthesis for networked systems with dynamic and unknown topology. Such networked systems arise in applications such as the power grid when lines between consumers are downe…
Deep Graph Matching Consensus
This work presents a two-stage neural architecture for learning and refining structural correspondences between graphs. First, we use localized node embeddings computed by a graph neural network to obtain an initial rank…
Entity AlignmentGraph MatchingGraph Neural NetworkKnowledge Graphs+1