Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model
Because it is not feasible to collect training data for every language, there is a growing interest in cross-lingual transfer learning. In this paper, we systematically explore zero-shot cross-lingual transfer learning on reading comprehension tasks with a language representation model pre-trained on multi-lingual corpus. The experimental results show that with pre-trained language representation zero-shot learning is feasible, and translating the source data into the target language is not necessary and even degrades the performance. We further explore what does the model learn in zero-shot setting.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual TransferReading ComprehensionTransfer LearningZero-Shot Cross-Lingual TransferZero-Shot LearningSimilar Papers 제목 키워드 기반
Improving Cross-Lingual Reading Comprehension with Self-Training
Substantial improvements have been made in machine reading comprehension, where the machine answers questions based on a given context. Current state-of-the-art models even surpass human performance on several benchmarks…
Machine Reading ComprehensionReading Comprehension2M-BELEBELE: Highly Multilingual Speech and American Sign Language Comprehension Dataset
We introduce the first highly multilingual speech and American Sign Language (ASL) comprehension dataset by extending BELEBELE. Our dataset covers 74 spoken languages at the intersection of BELEBELE and FLEURS, and one s…
BelebeleReading ComprehensionLearning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension
Multilingual pre-trained models are able to zero-shot transfer knowledge from rich-resource to low-resource languages in machine reading comprehension (MRC). However, inherent linguistic discrepancies in different langua…
Cross-Lingual TransferDisentanglementMachine Reading ComprehensionReading Comprehension+1Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension
Although many large-scale knowledge bases (KBs) claim to contain multilingual information, their support for many non-English languages is often incomplete. This incompleteness gives birth to the task of cross-lingual qu…
Cross-Lingual Question AnsweringMachine Reading ComprehensionQuestion AnsweringReading ComprehensionBilingual Alignment Pre-Training for Zero-Shot Cross-Lingual Transfer
Multilingual pre-trained models have achieved remarkable performance on cross-lingual transfer learning. Some multilingual models such as mBERT, have been pre-trained on unlabeled corpora, therefore the embeddings of dif…
Cross-Lingual TransferLanguage ModellingMachine Reading ComprehensionReading Comprehension+2