JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering
Existing KG-augmented models for commonsense question answering primarily focus on designing elaborate Graph Neural Networks (GNNs) to model knowledge graphs (KGs). However, they ignore (i) the effectively fusing and reasoning over question context representations and the KG representations, and (ii) automatically selecting relevant nodes from the noisy KGs during reasoning. In this paper, we propose a novel model, JointLK, which solves the above limitations through the joint reasoning of LM and GNN and the dynamic KGs pruning mechanism. Specifically, JointLK performs joint reasoning between LM and GNN through a novel dense bidirectional attention module, in which each question token attends on KG nodes and each KG node attends on question tokens, and the two modal representations fuse and update mutually by multi-step interactions. Then, the dynamic pruning module uses the attention weights generated by joint reasoning to prune irrelevant KG nodes recursively. We evaluate JointLK on the CommonsenseQA and OpenBookQA datasets, and demonstrate its improvements to the existing LM and LM+KG models, as well as its capability to perform interpretable reasoning.
Code (1)
Tasks
Knowledge GraphsQuestion AnsweringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering
Existing KG-augmented models for question answering primarily focus on designing elaborate Graph Neural Networks (GNNs) to model knowledge graphs (KGs). However, they ignore (i) the effectively fusing and reasoning over …
Knowledge GraphsQuestion AnsweringLanguage Model-Guided Knowledge Subgraphs for Question Answering
Knowledge graphs for question answering can provide subgraphs based on different combinations of questions and answers for multiple reasoning chains, in which humans often find the answer for a question. In this p…
Knowledge GraphsLanguage ModelingLanguage Modellingmodel+1QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
The problem of answering questions using knowledge from pre-trained language models (LMs) and knowledge graphs (KGs) presents two challenges: given a QA context (question and answer choice), methods need to (i) identify …
Common Sense ReasoningGraph Representation LearningKnowledge GraphsLanguage Modelling+5A Corpus for Reasoning About Natural Language Grounded in Photographs
We introduce a new dataset for joint reasoning about natural language and images, with a focus on semantic diversity, compositionality, and visual reasoning challenges. The data contains 107,292 examples of English sente…
DiversityVisual ReasoningTowards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs
Inspired by the success of large language models, there is a trend toward developing graph foundation models to conduct diverse downstream tasks in various domains. However, current models often require extra fine-tuning…
Knowledge GraphsZero-Shot Learning