WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing
Knowledge editing aims to rectify inaccuracies in large language models (LLMs) without costly retraining for outdated or erroneous knowledge. However, current knowledge editing methods primarily focus on single editing, failing to meet the requirements for lifelong editing. This study reveals a performance degradation encountered by knowledge editing in lifelong editing, characterized by toxicity buildup and toxicity flash, with the primary cause identified as pattern unmatch. We introduce a knowledge editing approach named Wise-Layer Knowledge Editor (WilKE), which selects editing layer based on the pattern matching degree of editing knowledge across different layers in language models. Experimental results demonstrate that, in lifelong editing, WilKE exhibits an average improvement of 46.2% and 67.8% on editing GPT2-XL and GPT-J relative to state-of-the-art knowledge editing methods.
Code (1)
Tasks
knowledge editingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Lifelong Knowledge Editing for Vision Language Models with Low-Rank Mixture-of-Experts
Model editing aims to correct inaccurate knowledge, update outdated information, and incorporate new data into Large Language Models (LLMs) without the need for retraining. This task poses challenges in lifelong scenario…
knowledge editingMixture-of-ExpertsModel EditingHoReN: Normalized Hopfield Retrieval for Large-Scale Sequential Model Editing
Large language models encode vast factual knowledge that can become outdated or incorrect after deployment, yet retraining is prohibitively costly. This motivates lifelong model editing, which updates targeted behavior w…
HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning
Lifelong model editing (LME) aims to sequentially rectify outdated or inaccurate knowledge in deployed LLMs while minimizing side effects on unrelated inputs. However, existing approaches typically apply parameter pertur…
Hierarchical Reinforcement LearningCross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Models
Multimodal Large Language Models (MLLMs) extend foundation models to real-world applications by integrating inputs such as text and vision. However, their broad knowledge capacity raises growing concerns about privacy le…
General KnowledgeUniAdapt: A Universal Adapter for Knowledge Calibration
Large Language Models (LLMs) require frequent updates to correct errors and keep pace with continuously evolving knowledge in a timely and effective manner. Recent research in it model editing has highlighted the challen…
Mixture-of-ExpertsModel EditingRetrieval-augmented GenerationSemantic Similarity+1