Dynamic Injection of Entity Knowledge into Dense Retrievers
Dense retrievers often struggle with queries involving less-frequent entities due to their limited entity knowledge. We propose the Knowledgeable Passage Retriever (KPR), a BERT-based retriever enhanced with a context-entity attention layer and dynamically updatable entity embeddings. This design enables KPR to incorporate external entity knowledge without retraining. Experiments on three datasets demonstrate that KPR consistently improves retrieval accuracy, with particularly large gains on the EntityQuestions dataset. When built on the off-the-shelf bge-base retriever, KPR achieves state-of-the-art performance among similarly sized models on two datasets. Models and code are released at https://github.com/knowledgeable-embedding/knowledgeable-embedding.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Few-Shot Generative Model Adaption via Identity Injection and Preservation
Training generative models with limited data presents severe challenges of mode collapse. A common approach is to adapt a large pretrained generative model upon a target domain with very few samples (fewer than 10), know…
Enhancing Document-level Relation Extraction by Entity Knowledge Injection
Document-level relation extraction (RE) aims to identify the relations between entities throughout an entire document. It needs complex reasoning skills to synthesize various knowledge such as coreferences and commonsens…
Document-level Relation ExtractionKnowledge GraphsRelationRelation ExtractionImproving and Diagnosing Knowledge-Based Visual Question Answering via Entity Enhanced Knowledge Injection
Knowledge-Based Visual Question Answering (KBVQA) is a bi-modal task requiring external world knowledge in order to correctly answer a text question and associated image. Recent single modality text work has shown knowle…
Common Sense ReasoningKnowledge Graph EmbeddingsQuestion AnsweringVisual Question Answering+2KILM: Knowledge Injection into Encoder-Decoder Language Models
Large pre-trained language models (PLMs) have been shown to retain implicit knowledge within their parameters. To enhance this implicit knowledge, we propose Knowledge Injection into Language Models (KILM), a novel appro…
DecoderEntity DisambiguationKEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning
Knowledge-enhanced pre-trained language models (KEPLMs) leverage relation triples from knowledge graphs (KGs) and integrate these external data sources into language models via self-supervised learning. Previous works tr…
Hierarchical Reinforcement LearningKnowledge GraphsNatural Language Understandingreinforcement-learning+3