Adversarial Training for Machine Reading Comprehension with Virtual Embeddings
Adversarial training (AT) as a regularization method has proved its effectiveness on various tasks. Though there are successful applications of AT on some NLP tasks, the distinguishing characteristics of NLP tasks have not been exploited. In this paper, we aim to apply AT on machine reading comprehension (MRC) tasks. Furthermore, we adapt AT for MRC tasks by proposing a novel adversarial training method called PQAT that perturbs the embedding matrix instead of word vectors. To differentiate the roles of passages and questions, PQAT uses additional virtual P/Q-embedding matrices to gather the global perturbations of words from passages and questions separately. We test the method on a wide range of MRC tasks, including span-based extractive RC and multiple-choice RC. The results show that adversarial training is effective universally, and PQAT further improves the performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine Reading ComprehensionMultiple-choiceReading ComprehensionSimilar Papers 제목 키워드 기반
Improving Machine Reading Comprehension via Adversarial Training
Adversarial training (AT) as a regularization method has proved its effectiveness in various tasks, such as image classification and text classification. Though there are successful applications of AT in many tasks of na…
General Classificationimage-classificationImage ClassificationMachine Reading Comprehension+3Stochastic Answer Networks for Machine Reading Comprehension
We propose a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension. Compared to previous work such as ReasoNet which used reinforcement learning to determi…
Machine Reading ComprehensionQuestion AnsweringReading Comprehensionreinforcement-learning+2Adversarial reading networks for machine comprehension
Machine reading has recently shown remarkable progress thanks to differentiable reasoning models. In this context, End-to-End trainable Memory Networks (MemN2N) have demonstrated promising performance on simple natural l…
Machine Reading ComprehensionQuestion AnsweringReading ComprehensionPower in Numbers: Robust reading comprehension by finetuning with four adversarial sentences per example
Recent models have achieved human level performance on the Stanford Question Answering Dataset when using F1 scores to evaluate the reading comprehension task. Yet, teaching machines to comprehend text has not been solve…
Question AnsweringReading ComprehensionSentenceDataset for the First Evaluation on Chinese Machine Reading Comprehension
Machine Reading Comprehension (MRC) has become enormously popular recently and has attracted a lot of attention. However, existing reading comprehension datasets are mostly in English. To add diversity in reading compreh…
Chinese Reading ComprehensionDiversityMachine Reading ComprehensionReading Comprehension