Knowledge Swapping via Learning and Unlearning
We introduce \textbf{Knowledge Swapping}, a novel task designed to selectively regulate knowledge of a pretrained model by enabling the forgetting of user\-specified information, retaining essential knowledge, and acquiring new knowledge simultaneously. By delving into the analysis of knock-on feature hierarchy, we find that incremental learning typically progresses from low\-level representations to higher\-level semantics, whereas forgetting tends to occur in the opposite direction\-starting from high-level semantics and moving down to low-level features. Building upon this, we propose to benchmark the knowledge swapping task with the strategy of \textit{Learning Before Forgetting}. Comprehensive experiments on various tasks like image classification, object detection, and semantic segmentation validate the effectiveness of the proposed strategy. The source code is available at \href{https://github.com/xingmingyu123456/KnowledgeSwapping}{https://github.com/xingmingyu123456/KnowledgeSwapping}.
Code (1)
Tasks
image-classificationImage ClassificationIncremental Learningobject-detectionObject DetectionSemantic SegmentationSimilar Papers 제목 키워드 기반
Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries
Evaluations of unlearning on reasoning models sometimes show a bypass pattern. The answer side looks unlearned, but the model's own thinking trace keeps emitting the forgotten content, and the gap is taken as evidence th…
Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter
Federated Learning is a promising paradigm for privacy-preserving collaborative model training. In practice, it is essential not only to continuously train the model to acquire new knowledge but also to guarantee old…
Federated LearningPrivacy PreservingERM-KTP: Knowledge-Level Machine Unlearning via Knowledge Transfer
Machine unlearning can fortify the privacy and security of machine learning applications. Unfortunately, the exact unlearning approaches are inefficient, and the approximate unlearning approaches are unsuitable for c…
Machine UnlearningTransfer LearningDoes Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Models
Large language models often retain unintended content, prompting growing interest in knowledge unlearning. Recent approaches emphasize localized unlearning, which restricts parameter updates to specific regions in an eff…
General KnowledgeEditing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?
Large language Model (LLM) unlearning, i.e., selectively removing information from LLMs, is vital for responsible model deployment. Differently, LLM knowledge editing aims to modify LLM knowledge instead of removing it. …
In-Context Learningknowledge editingLanguage ModelingLanguage Modelling+1