Empowering Language Models with Knowledge Graph Reasoning for Question Answering
Answering open-domain questions requires world knowledge about in-context entities. As pre-trained Language Models (LMs) lack the power to store all required knowledge, external knowledge sources, such as knowledge graphs, are often used to augment LMs. In this work, we propose knOwledge REasOning empowered Language Model (OREO-LM), which consists of a novel Knowledge Interaction Layer that can be flexibly plugged into existing Transformer-based LMs to interact with a differentiable Knowledge Graph Reasoning module collaboratively. In this way, LM guides KG to walk towards the desired answer, while the retrieved knowledge improves LM. By adopting OREO-LM to RoBERTa and T5, we show significant performance gain, achieving state-of-art results in the Closed-Book setting. The performance enhancement is mainly from the KG reasoning's capacity to infer missing relational facts. In addition, OREO-LM provides reasoning paths as rationales to interpret the model's decision.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge GraphsLanguage ModelingLanguage ModellingQuestion AnsweringWorld KnowledgeMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoning
Large Language Models (LLMs) achieve excellent performance in natural language reasoning tasks through pre-training on vast unstructured text, enabling them to understand the logic in natural language and generate logic-…
Graph Question AnsweringKnowledge GraphsA Pretraining Numerical Reasoning Model for Ordinal Constrained Question Answering on Knowledge Base
Knowledge Base Question Answering (KBQA) is to answer natural language questions posed over knowledge bases (KBs). This paper targets at empowering the IR-based KBQA models with the ability of numerical reasoning for ans…
Knowledge Base Question AnsweringQuestion AnsweringKG-Hopper: Empowering Compact Open LLMs with Knowledge Graph Reasoning via Reinforcement Learning
Large Language Models (LLMs) demonstrate impressive natural language capabilities but often struggle with knowledge-intensive reasoning tasks. Knowledge Base Question Answering (KBQA), which leverages structured Knowledg…
Knowledge Base Question AnsweringReinforcement LearningKnowledge GraphsEmpowering GraphRAG with Knowledge Filtering and Integration
In recent years, large language models (LLMs) have revolutionized the field of natural language processing. However, they often suffer from knowledge gaps and hallucinations. Graph retrieval-augmented generation (GraphRA…
Retrieval-augmented GenerationFind The Gap: Knowledge Base Reasoning For Visual Question Answering
We analyze knowledge-based visual question answering, for which given a question, the models need to ground it into the visual modality and retrieve the relevant knowledge from a given large knowledge base (KB) to be abl…
Question AnsweringRetrievalVisual Question AnsweringVisual Question Answering (VQA)