MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
We present the results of the Machine Reading for Question Answering (MRQA) 2019 shared task on evaluating the generalization capabilities of reading comprehension systems. In this task, we adapted and unified 18 distinct question answering datasets into the same format. Among them, six datasets were made available for training, six datasets were made available for development, and the final six were hidden for final evaluation. Ten teams submitted systems, which explored various ideas including data sampling, multi-task learning, adversarial training and ensembling. The best system achieved an average F1 score of 72.5 on the 12 held-out datasets, 10.7 absolute points higher than our initial baseline based on BERT.
Code (1)
Tasks
Multi-Task LearningQuestion AnsweringReading ComprehensionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension
In this paper, we introduce a simple system Baidu submitted for MRQA (Machine Reading for Question Answering) 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models. Our system is b…
Machine Reading ComprehensionMulti-Task LearningQuestion AnsweringReading ComprehensionCLER: Cross-task Learning with Expert Representation to Generalize Reading and Understanding
This paper describes our model for the reading comprehension task of the MRQA shared task. We propose CLER, which stands for Cross-task Learning with Expert Representation for the generalization of reading and understand…
Mixture-of-ExpertsMulti-Task LearningReading ComprehensionAn Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering
To produce a domain-agnostic question answering model for the Machine Reading Question Answering (MRQA) 2019 Shared Task, we investigate the relative benefits of large pre-trained language models, various data sampling s…
Data AugmentationQuestion AnsweringReading ComprehensionTranslationClinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset
Machine reading comprehension has made great progress in recent years owing to large-scale annotated datasets. In the clinical domain, however, creating such datasets is quite difficult due to the domain expertise requir…
Machine Reading ComprehensionQuestion AnsweringReading ComprehensionemrQA-msquad: A Medical Dataset Structured with the SQuAD V2.0 Framework, Enriched with emrQA Medical Information
Machine Reading Comprehension (MRC) holds a pivotal role in shaping Medical Question Answering Systems (QAS) and transforming the landscape of accessing and applying medical information. However, the inherent challenges …
Decision MakingMachine Reading ComprehensionMedical Question AnsweringQuestion Answering+1