ARES: A Reading Comprehension Ensembling Service
We introduce ARES (A Reading Comprehension Ensembling Service): a novel Machine Reading Comprehension (MRC) demonstration system which utilizes an ensemble of models to increase F1 by 2.3 points. While many of the top leaderboard submissions in popular MRC benchmarks such as the Stanford Question Answering Dataset (SQuAD) and Natural Questions (NQ) use model ensembles, the accompanying papers do not publish their ensembling strategies. In this work, we detail and evaluate various ensembling strategies using the NQ dataset. ARES leverages the CFO (Chakravarti et al., 2019) and ReactJS distributed frameworks to provide a scalable interactive Question Answering experience that capitalizes on the agreement (or lack thereof) between models to improve the answer visualization experience.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine Reading ComprehensionNatural QuestionsQuestion AnsweringReading ComprehensionSimilar Papers 제목 키워드 기반
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
We present the results of the Machine Reading for Question Answering (MRQA) 2019 shared task on evaluating the generalization capabilities of reading comprehension systems. In this task, we adapted and unified 18 distinc…
Multi-Task LearningQuestion AnsweringReading ComprehensionLearning to Search in Long Documents Using Document Structure
Reading comprehension models are based on recurrent neural networks that sequentially process the document tokens. As interest turns to answering more complex questions over longer documents, sequential reading of large …
Information RetrievalQuestion AnsweringReading ComprehensionRetrievalIdentifying Where to Focus in Reading Comprehension for Neural Question Generation
A first step in the task of automatically generating questions for testing reading comprehension is to identify \textit{question-worthy} sentences, i.e. sentences in a text passage that humans find it worthwhile to ask q…
Dependency ParsingMachine TranslationNamed Entity Recognition (NER)Question Generation+6TorontoCL at CMCL 2021 Shared Task: RoBERTa with Multi-Stage Fine-Tuning for Eye-Tracking Prediction
Eye movement data during reading is a useful source of information for understanding language comprehension processes. In this paper, we describe our submission to the CMCL 2021 shared task on predicting human reading pa…
regressionContinuous fluency tracking and the challenges of varying text complexity
This paper is a preliminary report on using text complexity measurement in the service of a new educational application. We describe a reading intervention where a child takes turns reading a book aloud with a virtual re…
Reading Comprehension