Ruminating Reader: Reasoning with Gated Multi-Hop Attention
To answer the question in machine comprehension (MC) task, the models need to establish the interaction between the question and the context. To tackle the problem that the single-pass model cannot reflect on and correct its answer, we present Ruminating Reader. Ruminating Reader adds a second pass of attention and a novel information fusion component to the Bi-Directional Attention Flow model (BiDAF). We propose novel layer structures that construct an query-aware context vector representation and fuse encoding representation with intermediate representation on top of BiDAF model. We show that a multi-hop attention mechanism can be applied to a bi-directional attention structure. In experiments on SQuAD, we find that the Reader outperforms the BiDAF baseline by a substantial margin, and matches or surpasses the performance of all other published systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringReading ComprehensionSimilar Papers 제목 키워드 기반
Gated-Attention Readers for Text Comprehension
In this paper we study the problem of answering cloze-style questions over documents. Our model, the Gated-Attention (GA) Reader, integrates a multi-hop architecture with a novel attention mechanism, which is based on mu…
Answer SelectionOpen-Domain Question AnsweringQuestion AnsweringReading ComprehensionRead and Comprehend by Gated-Attention Reader with More Belief
Gated-Attention (GA) Reader has been effective for reading comprehension. GA Reader makes two assumptions: (1) a uni-directional attention that uses an input query to gate token encodings of a document; (2) encoding at t…
PositionReading ComprehensionWord AlignmentClueReader: Heterogeneous Graph Attention Network for Multi-hop Machine Reading Comprehension
Multi-hop machine reading comprehension is a challenging task in natural language processing as it requires more reasoning ability across multiple documents. Spectral models based on graph convolutional networks have sho…
Graph AttentionMachine Reading ComprehensionReading ComprehensionMulti-step Retriever-Reader Interaction for Scalable Open-domain Question Answering
This paper introduces a new framework for open-domain question answering in which the retriever and the reader iteratively interact with each other. The framework is agnostic to the architecture of the machine reading mo…
Open-Domain Question AnsweringQuestion AnsweringReading ComprehensionTriviaQAForceReader: a BERT-based Interactive Machine Reading Comprehension Model with Attention Separation
The release of BERT revolutionized the development of NLP. Various BERT-based reading comprehension models have been proposed, thus updating the performance ranking of reading comprehension tasks. However, the above BERT…
Machine Reading ComprehensionReading Comprehension