One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval

2021-07-26 · NeurIPS 2021 12 · Akari Asai, Xinyan Yu, Jungo Kasai, Hannaneh Hajishirzi

We present Cross-lingual Open-Retrieval Answer Generation (CORA), the first unified many-to-many question answering (QA) model that can answer questions across many languages, even for ones without language-specific annotated data or knowledge sources. We introduce a new dense passage retrieval algorithm that is trained to retrieve documents across languages for a question. Combined with a multilingual autoregressive generation model, CORA answers directly in the target language without any translation or in-language retrieval modules as used in prior work. We propose an iterative training method that automatically extends annotated data available only in high-resource languages to low-resource ones. Our results show that CORA substantially outperforms the previous state of the art on multilingual open QA benchmarks across 26 languages, 9 of which are unseen during training. Our analyses show the significance of cross-lingual retrieval and generation in many languages, particularly under low-resource settings.

📄 PDF Abstract BibTeX arXiv:2107.11976

Code (1)

AkariAsai/CORA 공식 구현 pytorch

Tasks

Answer Generation Passage Retrieval Question Answering Retrieval Translation

XOR QA: Cross-lingual Open-Retrieval Question Answering

2020-10-22 · NAACL 2021 4 · Akari Asai, Jungo Kasai, Jonathan H. Clark, Kenton Lee 외

Multilingual question answering tasks typically assume answers exist in the same language as the question. Yet in practice, many languages face both information scarcity -- where languages have few reference articles -- …

Articles Machine Translation Question Answering Retrieval+1

Bridging the Language Gap: Knowledge Injected Multilingual Question Answering

2023-04-06 · Zhichao Duan, Xiuxing Li, Zhengyan Zhang, Zhenyu Li 외

Question Answering (QA) is the task of automatically answering questions posed by humans in natural languages. There are different settings to answer a question, such as abstractive, extractive, boolean, and multiple-cho…

Cross-Lingual Transfer Extractive Question-Answering Link Prediction Multiple-choice+1

XQA: A Cross-lingual Open-domain Question Answering Dataset

2019-07-01 · ACL 2019 7 · Jiahua Liu, Yankai Lin, Zhiyuan Liu, Maosong Sun

Open-domain question answering (OpenQA) aims to answer questions through text retrieval and reading comprehension. Recently, lots of neural network-based models have been proposed and achieved promising results in OpenQA…

Machine Translation Open-Domain Question Answering Question Answering Reading Comprehension+2

Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension

2023-02-26 · Chen Zhang, Yuxuan Lai, Yansong Feng, Xingyu Shen 외

Although many large-scale knowledge bases (KBs) claim to contain multilingual information, their support for many non-English languages is often incomplete. This incompleteness gives birth to the task of cross-lingual qu…

Cross-Lingual Question Answering Machine Reading Comprehension Question Answering Reading Comprehension

A Survey on non-English Question Answering Dataset

2021-12-27 · Andreas Chandra, Affandy Fahrizain, Ibrahim, Simon Willyanto Laufried

Research in question answering datasets and models has gained a lot of attention in the research community. Many of them release their own question answering datasets as well as the models. There is tremendous progress t…

Cross-Lingual Question Answering Question Answering Survey