When Shared Knowledge Hurts: Spectral Over-Accumulation in Model Merging

2026-02-05 · Yayuan Li, Ze Peng, Jian Zhang, Jintao Guo, Yue Duan, Yinghuan Shi arxiv

Model merging combines multiple fine-tuned models into a single model by adding their weight updates, providing a lightweight alternative to retraining. Existing methods primarily target resolving conflicts between task updates, leaving the failure mode of over-counting shared knowledge unaddressed. We show that when tasks share aligned spectral directions (i.e., overlapping singular vectors), a simple linear combination repeatedly accumulates these directions, inflating the singular values and biasing the merged model toward shared subspaces. To mitigate this issue, we propose Singular Value Calibration (SVC), a training-free and data-free post-processing method that quantifies subspace overlap and rescales inflated singular values to restore a balanced spectrum. Across vision and language benchmarks, SVC consistently improves strong merging baselines and achieves state-of-the-art performance. Furthermore, by modifying only the singular values, SVC improves the performance of Task Arithmetic by 13.0%. Code is available at https://github.com/lyymuwu/SVC.

📄 PDF Abstract BibTeX arXiv:2602.05536

Code (0)

등록된 구현이 없습니다.

Multitask Deep Learning with Spectral Knowledge for Hyperspectral Image Classification

2019-05-11 · Sheng-Jie Liu, Qian Shi

In this letter, we propose a multitask deep learning method for classification of multiple hyperspectral data in a single training. Deep learning models have achieved promising results on hyperspectral image classificati…

Classification Deep Learning General Classification Hyperspectral Image Classification+2

Improving Multi-Task Generalization via Regularizing Spurious Correlation

2022-05-19 · Ziniu Hu, Zhe Zhao, Xinyang Yi, Tiansheng Yao 외

Multi-Task Learning (MTL) is a powerful learning paradigm to improve generalization performance via knowledge sharing. However, existing studies find that MTL could sometimes hurt generalization, especially when two task…

Multi-Task Learning Representation Learning

When Covariate-shifted Data Augmentation Increases Test Error And How to Fix It

2019-09-25 · Sang Michael Xie*, Aditi Raghunathan*, Fanny Yang, John C. Duchi 외

Empirically, data augmentation sometimes improves and sometimes hurts test error, even when only adding points with labels from the true conditional distribution that the hypothesis class is expressive enough to fit. In…

Data Augmentation regression

Lifelong Spectral Clustering

2019-11-27 · Gan Sun, Yang Cong, Qianqian Wang, Jun Li 외

In the past decades, spectral clustering (SC) has become one of the most effective clustering algorithms. However, most previous studies focus on spectral clustering tasks with a fixed task set, which cannot incorporate …

Clustering

Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks

2019-11-01 · WS 2019 11 · Xiepeng Li, Zhexi Zhang, Wei Zhu, Zheng Li 외

To solve the shared tasks of COIN: COmmonsense INference in Natural Language Processing) Workshop in , we need explore the impact of knowledge representation in modeling commonsense knowledge to boost performance of mach…

Common Sense Reasoning Machine Reading Comprehension Reading Comprehension Task 2+1