paper-with-me

홈 › Papers

XCMRC: Evaluating Cross-lingual Machine Reading Comprehension

2019-08-15 · Pengyuan Liu, Yuning Deng, Chenghao Zhu, Han Hu

We present XCMRC, the first public cross-lingual language understanding (XLU) benchmark which aims to test machines on their cross-lingual reading comprehension ability. To be specific, XCMRC is a Cross-lingual Cloze-style Machine Reading Comprehension task which requires the reader to fill in a missing word (we additionally provide ten noun candidates) in a sentence written in target language (English / Chinese) by reading a given passage written in source language (Chinese / English). Chinese and English are rich-resource language pairs, in order to study low-resource cross-lingual machine reading comprehension (XMRC), besides defining the common XCMRC task which has no restrictions on use of external language resources, we also define the pseudo low-resource XCMRC task by limiting the language resources to be used. In addition, we provide two baselines for common XCMRC task and two for pseudo XCMRC task respectively. We also provide an upper bound baseline for both tasks. We found that for common XCMRC task, translation-based method and multilingual sentence encoder-based method can obtain reasonable performance but still have much room for improvement. As for pseudo low-resource XCMRC task, due to strict restrictions on the use of language resources, our two approaches are far below the upper bound so there are many challenges ahead.

📄 PDF Abstract BibTeX arXiv:1908.05416

Code (0)

등록된 구현이 없습니다.

Tasks

Machine Reading ComprehensionReading ComprehensionSentence

Similar Papers 제목 키워드 기반

Evaluating Machine Reading Systems through Comprehension Tests

2012-05-01 · LREC 2012 5 · Anselmo Pe{\~n}as, Eduard Hovy, Pamela Forner, {\'A}lvaro Rodrigo 외

This paper describes a methodology for testing and evaluating the performance of Machine Reading systems through Question Answering and Reading Comprehension Tests. The methodology is being used in QA4MRE (QA for Machine…

Answer SelectionMultiple-choiceQuestion AnsweringReading Comprehension

Improving Cross-Lingual Reading Comprehension with Self-Training

2021-05-08 · Wei-Cheng Huang, Chien-yu Huang, Hung-Yi Lee

Substantial improvements have been made in machine reading comprehension, where the machine answers questions based on a given context. Current state-of-the-art models even surpass human performance on several benchmarks…

Machine Reading ComprehensionReading Comprehension

Cross-lingual and Cross-domain Evaluation of Machine Reading Comprehension with Squad and CALOR-Quest Corpora

2020-05-01 · LREC 2020 5 · Delphine Charlet, Geraldine Damnati, Frederic Bechet, Gabriel Marzinotto 외

Machine Reading received recently a lot of attention thanks to both the availability of very large corpora such as SQuAD or MS MARCO containing triplets (document, question, answer), and the introduction of Transformer L…

Machine Reading ComprehensionReading Comprehension

Cross-Lingual Machine Reading Comprehension

2019-09-01 · IJCNLP 2019 11 · Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin 외

Though the community has made great progress on Machine Reading Comprehension (MRC) task, most of the previous works are solving English-based MRC problems, and there are few efforts on other languages mainly due to the …

Machine Reading ComprehensionReading ComprehensionTranslation

Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation

2020-10-27 · COLING 2020 8 · Junhao Liu, Linjun Shou, Jian Pei, Ming Gong 외

Cross-lingual Machine Reading Comprehension (CLMRC) remains a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi, and Vietnamese. Many previous approaches…

Knowledge DistillationMachine Reading ComprehensionReading ComprehensionTranslation