Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation
Cross-lingual Machine Reading Comprehension (CLMRC) remains a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi, and Vietnamese. Many previous approaches use translation data by translating from a rich-source language, such as English, to low-source languages as auxiliary supervision. However, how to effectively leverage translation data and reduce the impact of noise introduced by translation remains onerous. In this paper, we tackle this challenge and enhance the cross-lingual transferring performance by a novel augmentation approach named Language Branch Machine Reading Comprehension (LBMRC). A language branch is a group of passages in one single language paired with questions in all target languages. We train multiple machine reading comprehension (MRC) models proficient in individual language based on LBMRC. Then, we devise a multilingual distillation approach to amalgamate knowledge from multiple language branch models to a single model for all target languages. Combining the LBMRC and multilingual distillation can be more robust to the data noises, therefore, improving the model's cross-lingual ability. Meanwhile, the produced single multilingual model is applicable to all target languages, which saves the cost of training, inference, and maintenance for multiple models. Extensive experiments on two CLMRC benchmarks clearly show the effectiveness of our proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationMachine Reading ComprehensionReading ComprehensionTranslationSimilar Papers 제목 키워드 기반
Improving Cross-Lingual Reading Comprehension with Self-Training
Substantial improvements have been made in machine reading comprehension, where the machine answers questions based on a given context. Current state-of-the-art models even surpass human performance on several benchmarks…
Machine Reading ComprehensionReading ComprehensionXCMRC: Evaluating Cross-lingual Machine Reading Comprehension
We present XCMRC, the first public cross-lingual language understanding (XLU) benchmark which aims to test machines on their cross-lingual reading comprehension ability. To be specific, XCMRC is a Cross-lingual Cloze-sty…
Machine Reading ComprehensionReading ComprehensionSentenceCross-Lingual Machine Reading Comprehension
Though the community has made great progress on Machine Reading Comprehension (MRC) task, most of the previous works are solving English-based MRC problems, and there are few efforts on other languages mainly due to the …
Machine Reading ComprehensionReading ComprehensionTranslationCross-lingual and Cross-domain Evaluation of Machine Reading Comprehension with Squad and CALOR-Quest Corpora
Machine Reading received recently a lot of attention thanks to both the availability of very large corpora such as SQuAD or MS MARCO containing triplets (document, question, answer), and the introduction of Transformer L…
Machine Reading ComprehensionReading ComprehensionRevealing Weaknesses of Vietnamese Language Models Through Unanswerable Questions in Machine Reading Comprehension
Although the curse of multilinguality significantly restricts the language abilities of multilingual models in monolingual settings, researchers now still have to rely on multilingual models to develop state-of-the-art s…
Machine Reading ComprehensionReading ComprehensionVietnamese Language ModelsVietnamese Machine Reading Comprehension