Transporting Task Vectors across Different Architectures without Training
Adapting large pre-trained models to downstream tasks often produces task-specific parameter updates that are expensive to relearn for every model variant. While recent work has shown that such updates can be transferred between models with identical architectures, transferring them across models of different widths remains unexplored. In this work, we introduce Theseus, a training-free method for transporting task updates across heterogeneous-width models. Rather than matching parameters, we characterize a task update by the functional effect it induces on intermediate representations. We formalize task-vector transport as a functional matching problem on observed activations and show that, after aligning representation spaces via orthogonal Procrustes analysis, it admits a stable closed-form solution that preserves the geometry of the update. We evaluate Theseus on vision and language models across different widths, showing consistent improvements over baselines without additional training or backpropagation. Our results show that task updates can be meaningfully transferred across architectures when task identity is defined functionally rather than parametrically. Code is available at https://github.com/apanariello4/merge-and-rebase.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Gradient-Sign Masking for Task Vector Transport Across Pre-Trained Models
When a new release of a foundation model is published, practitioners typically need to repeat fine-tuning, even if the same task was already tackled in the previous version. A promising alternative is to reuse the parame…
Go with the Flow: the distribution of information processing in multi-path networks
The architectures of convolution neural networks (CNN) have a great impact on the predictive performance and efficiency of the model. Yet, the development of these architectures is still driven by trial and error, making…
Empirical Studies on the Convergence of Feature Spaces in Deep Learning
While deep learning is effective to learn features/representations from data, the distributions of samples in feature spaces learned by various architectures for different training tasks (e.g., latent layers of AEs and f…
Image ReconstructionSelf-Supervised LearningControllable Neural Architectures for Multi-Task Control
This paper studies a multi-task control problem where multiple linear systems are to be regulated by a single non-linear controller. In particular, motivated by recent advances in multi-task learning and the design of br…
Multi-Task LearningUnderstanding In-Context Learning Beyond Transformers: An Investigation of State Space and Hybrid Architectures
We perform in-depth evaluations of in-context learning (ICL) on state-of-the-art transformer, state-space, and hybrid large language models over two categories of knowledge-based ICL tasks. Using a combination of behavio…