Training Question Answering Models From Synthetic Data
Question and answer generation is a data augmentation method that aims to improve question answering (QA) models given the limited amount of human labeled data. However, a considerable gap remains between synthetic and human-generated question-answer pairs. This work aims to narrow this gap by taking advantage of large language models and explores several factors such as model size, quality of pretrained models, scale of data synthesized, and algorithmic choices. On the SQuAD1.1 question answering task, we achieve higher accuracy using solely synthetic questions and answers than when using the SQuAD1.1 training set questions alone. Removing access to real Wikipedia data, we synthesize questions and answers from a synthetic corpus generated by an 8.3 billion parameter GPT-2 model. With no access to human supervision and only access to other models, we are able to train state of the art question answering networks on entirely model-generated data that achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. We further apply our methodology to SQuAD2.0 and show a 2.8 absolute gain on EM score compared to prior work using synthetic data.
Code (0)
등록된 구현이 없습니다.
Tasks
Answer GenerationData AugmentationQuestion AnsweringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Synthetic Data Generation for Multilingual Domain-Adaptable Question Answering Systems
Deep learning models have significantly advanced the state of the art of question answering systems. However, the majority of datasets available for training such models have been annotated by humans, are open-domain, an…
Question AnsweringSynthetic Data GenerationSanitizing Synthetic Training Data Generation for Question Answering over Knowledge Graphs
Synthetic data generation is important to training and evaluating neural models for question answering over knowledge graphs. The quality of the data and the partitioning of the datasets into training, validation and tes…
Knowledge GraphsQuestion AnsweringSynthetic Data GenerationUnsupervised Adaptation of Question Answering Systems via Generative Self-training
BERT-era question answering systems have recently achieved impressive performance on several question-answering (QA) tasks. These systems are based on representations that have been pre-trained on self-supervised tasks s…
Question AnsweringSentenceSynthetic Question Value Estimation for Domain Adaptation of Question Answering
Synthesizing QA pairs with a question generator (QG) on the target domain has become a popular approach for domain adaptation of question answering (QA) models. Since synthetic questions are often noisy in practice, exis…
Domain AdaptationQuestion AnsweringSynthetic Question Value Estimation for Domain Adaptation of Question Answering
Synthesizing QA pairs with a question generator (QG) on the target domain has become a popular approach for domain adaptation of question answering (QA) models. Since synthetic questions are often noisy in practice, exis…
Domain AdaptationQuestion Answering