Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge
In this paper, we aim to extract commonsense knowledge to improve machine reading comprehension. We propose to represent relations implicitly by situating structured knowledge in a context instead of relying on a pre-defined set of relations, and we call it contextualized knowledge. Each piece of contextualized knowledge consists of a pair of interrelated verbal and nonverbal messages extracted from a script and the scene in which they occur as context to implicitly represent the relation between the verbal and nonverbal messages, which are originally conveyed by different modalities within the script. We propose a two-stage fine-tuning strategy to use the large-scale weakly-labeled data based on a single type of contextualized knowledge and employ a teacher-student paradigm to inject multiple types of contextualized knowledge into a student machine reader. Experimental results demonstrate that our method outperforms a state-of-the-art baseline by a 4.3% improvement in accuracy on the machine reading comprehension dataset C^3, wherein most of the questions require unstated prior knowledge.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine Reading ComprehensionReading ComprehensionSimilar Papers 제목 키워드 기반
Evaluating Commonsense in Pre-trained Language Models
Contextualized representations trained over large raw text data have given remarkable improvements for NLP tasks including question answering and reading comprehension. There have been works showing that syntactic, seman…
Language ModelingLanguage ModellingQuestion AnsweringReading ComprehensionNeural Network-based Models with Commonsense Knowledge for Machine Reading Comprehension
State-of-the-art machine reading comprehension models are capable of producing answers for factual questions about a given piece of text. However, some type of questions requires commonsense knowledge which cannot be inf…
Machine Reading ComprehensionReading ComprehensionCommonsense Knowledge + BERT for Level 2 Reading Comprehension Ability Test
Commonsense knowledge plays an important role when we read. The performance of BERT on SQuAD dataset shows that the accuracy of BERT can be better than human users. However, it does not mean that computers can surpass th…
ArticlesReading ComprehensionCommonsense knowledge adversarial dataset that challenges ELECTRA
Commonsense knowledge is critical in human reading comprehension. While machine comprehension has made significant progress in recent years, the ability in handling commonsense knowledge remains limited. Synonyms are one…
Reading ComprehensionWord Sense DisambiguationReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension
We present a large-scale dataset, ReCoRD, for machine reading comprehension requiring commonsense reasoning. Experiments on this dataset demonstrate that the performance of state-of-the-art MRC systems fall far behind hu…
Common Sense ReasoningMachine Reading ComprehensionReading Comprehension