XQA: A Cross-lingual Open-domain Question Answering Dataset
Open-domain question answering (OpenQA) aims to answer questions through text retrieval and reading comprehension. Recently, lots of neural network-based models have been proposed and achieved promising results in OpenQA. However, the success of these models relies on a massive volume of training data (usually in English), which is not available in many other languages, especially for those low-resource languages. Therefore, it is essential to investigate cross-lingual OpenQA. In this paper, we construct a novel dataset XQA for cross-lingual OpenQA research. It consists of a training set in English as well as development and test sets in eight other languages. Besides, we provide several baseline systems for cross-lingual OpenQA, including two machine translation-based methods and one zero-shot cross-lingual method (multilingual BERT). Experimental results show that the multilingual BERT model achieves the best results in almost all target languages, while the performance of cross-lingual OpenQA is still much lower than that of English. Our analysis indicates that the performance of cross-lingual OpenQA is related to not only how similar the target language and English are, but also how difficult the question set of the target language is. The XQA dataset is publicly available at http://github.com/thunlp/XQA.
Code (1)
Tasks
Machine TranslationOpen-Domain Question AnsweringQuestion AnsweringReading ComprehensionRetrievalText RetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains
Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains. However, low-resource settings such as new and emerging domains would especially benefit from reli…
ArticlesQuestion AnsweringRetrievalTranslationCross-Lingual Open-Domain Question Answering with Answer Sentence Generation
Open-Domain Generative Question Answering has achieved impressive performance in English by combining document-level retrieval with answer generation. These approaches, which we refer to as GenQA, can generate complete s…
Answer GenerationGenerative Question AnsweringOpen-Domain Question AnsweringQuestion Answering+2MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering
Progress in cross-lingual modeling depends on challenging, realistic, and diverse evaluation sets. We introduce Multilingual Knowledge Questions and Answers (MKQA), an open-domain question answering evaluation set compri…
Extractive Question-AnsweringNatural QuestionsOpen-Domain Question AnsweringQuestion Answering+1Investigating Information Inconsistency in Multilingual Open-Domain Question Answering
Retrieval based open-domain QA systems use retrieved documents and answer-span selection over retrieved documents to find best-answer candidates. We hypothesize that multilingual Question Answering (QA) systems are prone…
Open-Domain Question AnsweringQuestion AnsweringRetrievalMulti-Domain Multilingual Question Answering
Question answering (QA) is one of the most challenging and impactful tasks in natural language processing. Most research in QA, however, has focused on the open-domain or monolingual setting while most real-world applica…
Cross-Lingual TransferDomain AdaptationMulti-Task LearningQuestion Answering+1