Papers Task Arithmetic
“Task Arithmetic” 태그가 달린 논문 61편 · 필터 해제
Transferring Visual Explainability of Self-Explaining Models through Task Arithmetic
In scenarios requiring both prediction and explanation efficiency for image classification, self-explaining models that perform both tasks in a single inference are effective. However, their training incurs substantial l…
image-classificationImage ClassificationTask ArithmeticDuET: Dual Incremental Object Detection via Exemplar-Free Task Arithmetic
Real-world object detection systems, such as those in autonomous driving and surveillance, must continuously learn new object categories and simultaneously adapt to changing environmental conditions. Existing approaches,…
Autonomous DrivingAvgClass-Incremental Object DetectionExemplar-Free+5CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning
Recent advances in music foundation models have improved audio representation learning, yet their effectiveness across diverse musical traditions remains limited. We introduce CultureMERT-95M, a multi-culturally adapted …
Music Auto-TaggingRepresentation LearningTask ArithmeticSubspace-Boosted Model Merging
Model merging enables the combination of multiple specialized expert models into a single model capable of performing multiple tasks. However, the benefits of merging an increasing amount of specialized experts generally…
modelTask ArithmeticCALM: Consensus-Aware Localized Merging for Multi-Task Learning
Model merging aims to integrate the strengths of multiple fine-tuned models into a unified model while preserving task-specific capabilities. Existing methods, represented by task arithmetic, are typically classified int…
Multi-Task LearningTask ArithmeticFedRPCA: Enhancing Federated LoRA Aggregation Using Robust PCA
LoRA has emerged as one of the most promising fine-tuning techniques, especially for federated learning (FL), since it significantly reduces communication and computation costs at resource-constrained clients. However, d…
Federated LearningTask ArithmeticOn Fairness of Task Arithmetic: The Role of Task Vectors
Model editing techniques, particularly task arithmetic using task vectors, have shown promise in efficiently modifying pre-trained models through arithmetic operations like task addition and negation. Despite computation…
FairnessHate Speech DetectionModel EditingNegation+2Scalable Strategies for Continual Learning with Replay
Future deep learning models will be distinguished by systems that perpetually learn through interaction, imagination, and cooperation, blurring the line between training and inference. This makes continual learning a cri…
Continual LearningTask ArithmeticCross-Model Transfer of Task Vectors via Few-Shot Orthogonal Alignment
Task arithmetic enables efficient model editing by representing task-specific changes as vectors in parameter space. Task arithmetic typically assumes that the source and target models are initialized from the same pre-t…
Model EditingTask ArithmeticMCU: Improving Machine Unlearning through Mode Connectivity
Machine Unlearning (MU) aims to remove the information of specific training data from a trained model, ensuring compliance with privacy regulations and user requests. While one line of existing MU methods relies on linea…
image-classificationImage ClassificationMachine UnlearningTask ArithmeticCAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging
Multi-task model merging offers a promising paradigm for integrating multiple expert models into a unified model without additional training. Existing state-of-the-art techniques, such as Task Arithmetic and its variants…
Task ArithmeticInvestigating Task Arithmetic for Zero-Shot Information Retrieval
Large Language Models (LLMs) have shown impressive zero-shot performance across a variety of Natural Language Processing tasks, including document re-ranking. However, their effectiveness degrades on unseen tasks and dom…
Information RetrievalRe-RankingRetrievalTask Arithmetic+1Leveraging Submodule Linearity Enhances Task Arithmetic Performance in LLMs
Task arithmetic is a straightforward yet highly effective strategy for model merging, enabling the resultant model to exhibit multi-task capabilities. Recent research indicates that models demonstrating linearity enhance…
Task ArithmeticWhen is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers
Task arithmetic refers to editing the pre-trained model by adding a weighted sum of task vectors, each of which is the weight update from the pre-trained model to fine-tuned models for certain tasks. This approach recent…
Binary ClassificationDomain GeneralizationLarge Language ModelMachine Unlearning+4Single-Input Multi-Output Model Merging: Leveraging Foundation Models for Dense Multi-Task Learning
Model merging is a flexible and computationally tractable approach to merge single-task checkpoints into a multi-task model. Prior work has solely focused on constrained multi-task settings where there is a one-to-one ma…
Multi-Task LearningScene UnderstandingTask ArithmeticEfficient Model Editing with Task-Localized Sparse Fine-tuning
Task arithmetic has emerged as a promising approach for editing models by representing task-specific knowledge as composable task vectors. However, existing methods rely on network linearization to derive task vectors, l…
DisentanglementModel EditingNegationTask ArithmeticOpenThaiGPT 1.6 and R1: Thai-Centric Open Source and Reasoning Large Language Models
We present OpenThaiGPT 1.6 and R1 (OTG-1.6 and OTG-R1), Thai-centric Large Language Models (LLMs) developed through distinct methodologies to enhance generalization and reasoning capabilities. OTG-1.6 employs Task Arithm…
Task ArithmeticDisentangling Task Interference within Neurons: Model Merging in Alignment with Neuronal Mechanisms
Fine-tuning pre-trained models on targeted datasets enhances task-specific performance but often comes at the expense of generalization. Model merging techniques, which integrate multiple fine-tuned models into a single …
Task ArithmeticLayer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge
Large language models (LLMs) demonstrate strong task-specific capabilities through fine-tuning, but merging multiple fine-tuned models often leads to degraded performance due to overlapping instruction-following componen…
GSM8KHumanEvalInstruction FollowingMulti-Task Learning+1Neural Networks Remember More: The Power of Parameter Isolation and Combination
Catastrophic forgetting is a pervasive issue for pre-trained language models (PLMs) during continual learning, where models lose previously acquired knowledge when sequentially trained on a series of tasks. The model's a…
Continual LearningTask Arithmetic