K-Edit: Language Model Editing with Contextual Knowledge Awareness
As the world changes, we need to be able to update our models and correct false information without costly retraining. Knowledge-based model editing enables precise modifications to the weights of large language models in order to modify the information encoded within. Recent approaches have seen success in enabling recall of edited information for thousands of edits at once. However, these approaches fail to produce edits that account for associated contextual information. We present K-Edit, an effective approach to generating contextually consistent knowledge edits. By using knowledge graphs, which maintain contextual consistency when an edge is edited, we are able to generate additional \textit{contextual edits} that ensure consistency of related information in the language model. Our experiments demonstrate significant improvements in multi-hop question answering while maintaining the general effectiveness and scalability of model edits.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge GraphsLanguage ModelingLanguage ModellingModel EditingMulti-hop Question AnsweringQuestion AnsweringSimilar Papers 제목 키워드 기반
Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs
Knowledge editing enables multimodal large language models (MLLMs) to efficiently update outdated or incorrect information. However, existing benchmarks primarily emphasize cognitive-level modifications while lacking a f…
knowledge editingEvaluating Dependencies in Fact Editing for Language Models: Specificity and Implication Awareness
The potential of using a large language model (LLM) as a knowledge base (KB) has sparked significant interest. To manage the knowledge acquired by LLMs, we need to ensure that the editing of learned facts respects intern…
knowledge editingLanguage ModelingLanguage ModellingLarge Language Model+2Uncovering Context Reliance in Unstructured Knowledge Editing
Editing Large language models (LLMs) with real-world, unstructured knowledge is essential for correcting and updating their internal parametric knowledge. In this work, we revisit the fundamental next-token prediction (N…
knowledge editingRobust and Scalable Model Editing for Large Language Models
Large language models (LLMs) can make predictions using parametric knowledge--knowledge encoded in the model weights--or contextual knowledge--knowledge presented in the context. In many scenarios, a desirable behavior i…
Model EditingEVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries
The dynamic nature of real-world information necessitates efficient knowledge editing (KE) in large language models (LLMs) for knowledge updating. However, current KE approaches, which typically operate on (subject, rela…
knowledge editing