Synthetic Question Value Estimation for Domain Adaptation of Question Answering
Synthesizing QA pairs with a question generator (QG) on the target domain has become a popular approach for domain adaptation of question answering (QA) models. Since synthetic questions are often noisy in practice, existing work adapts scores from a pretrained QA (or QG) model as criteria to select high-quality questions. However, these scores do not directly serve the ultimate goal of improving QA performance on the target domain. In this paper, we introduce a novel idea of training a question value estimator (QVE) that directly estimates the usefulness of synthetic questions for improving the target-domain QA performance. By conducting comprehensive experiments, we show that the synthetic questions selected by QVE can help achieve better target-domain QA performance, in comparison with existing techniques. We additionally show that by using such questions and only around 15% of the human annotations on the target domain, we can achieve comparable performance to the fully-supervised baselines.
Code (1)
Tasks
Domain AdaptationQuestion AnsweringSimilar Papers 제목 키워드 기반
Synthetic Question Value Estimation for Domain Adaptation of Question Answering
Synthesizing QA pairs with a question generator (QG) on the target domain has become a popular approach for domain adaptation of question answering (QA) models. Since synthetic questions are often noisy in practice, exis…
Domain AdaptationQuestion AnsweringEnd-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems
We propose an end-to-end approach for synthetic QA data generation. Our model comprises a single transformer-based encoder-decoder network that is trained end-to-end to generate both answers and questions. In a nutshell,…
DecoderDomain AdaptationQuestion AnsweringSynthetic Data GenerationQA Domain Adaptation using Data Augmentation and Contrastive Adaptation
Domain adaptation for question answering (QA) has recently shown impressive results for answering out-of-domain questions. Yet, a common challenge is to build approaches that are effective for niche domains with small te…
Data AugmentationDomain AdaptationQuestion AnsweringQuestion Generation+1Contrastive Domain Adaptation for Question Answering using Limited Text Corpora
Question generation has recently shown impressive results in customizing question answering (QA) systems to new domains. These approaches circumvent the need for manually annotated training data from the new domain and, …
Domain AdaptationQuestion AnsweringQuestion GenerationQuestion-GenerationBack-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval
In this work, we introduce back-training, an alternative to self-training for unsupervised domain adaptation (UDA) from source to target domain. While self-training generates synthetic training data where natural inputs …
BIG-bench Machine LearningDomain AdaptationNatural QuestionsPassage Retrieval+4