Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction
We present a pioneering study on leveraging multilingual pre-trained generative language models for zero-shot cross-lingual event argument extraction (EAE) by formulating EAE as a language generation task. Compared to previous classification-based EAE models that build classifiers on top of pre-trained masked language models, our generative model effectively encodes the event structures and better captures the dependencies between arguments. To achieve cross-lingual transfer, we design language-agnostic templates to encode argument roles and train our models on source languages to "generate" arguments in the source languages to fill in the language-agnostic template. The trained model can then be directly applied to target languages to "generate" arguments in the target languages to fill in the template. Our experimental results demonstrate that the proposed model outperforms the current state-of-the-art results on zero-shot cross-lingual EAE. Comprehensive ablation study and error analysis are presented to better understand the advantages and the current limitations of using multilingual generative language models for cross-lingual transfer.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual TransferEvent Argument ExtractionText GenerationSimilar Papers 제목 키워드 기반
Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction
We present a study on leveraging multilingual pre-trained generative language models for zero-shot cross-lingual event argument extraction (EAE). By formulating EAE as a language generation task, our method effectively e…
Cross-Lingual TransferEvent Argument ExtractionText GenerationZero-Shot Cross-Lingual TransferTowards Zero-Shot Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension
We propose a simple method to generate multilingual question and answer pairs on a large scale through the use of a single generative model. These synthetic samples can be used to improve the zero-shot performance of mul…
Answer GenerationCross-Lingual Question AnsweringReading ComprehensionFew-shot Learning with Multilingual Language Models
Large-scale generative language models such as GPT-3 are competitive few-shot learners. While these models are known to be able to jointly represent many different languages, their training data is dominated by English, …
Cross-Lingual TransferFew-Shot LearningHate Speech DetectionMachine Translation+3Cross-Lingual Text Classification with Multilingual Distillation and Zero-Shot-Aware Training
Multilingual pre-trained language models (MPLMs) not only can handle tasks in different languages but also exhibit surprising zero-shot cross-lingual transferability. However, MPLMs usually are not able to achieve compar…
Language ModelingLanguage Modellingtext-classificationText ClassificationThe Multilingual Curse at the Retrieval Layer: Evidence from Amharic
Multilingual retrieval increasingly underpins cross-lingual question answering and retrieval-augmented generation. Strong zero-shot scores on multilingual benchmarks are often taken as evidence that current encoders tran…
Cross-Lingual Question AnsweringPassage Retrieval