Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages
Multilingual Pretrained Language Models (MPLMs) have shown their strong multilinguality in recent empirical cross-lingual transfer studies. In this paper, we propose the Prompts Augmented by Retrieval Crosslingually (PARC) pipeline to improve the zero-shot performance on low-resource languages (LRLs) by augmenting the context with semantically similar sentences retrieved from a high-resource language (HRL) as prompts. PARC improves the zero-shot performance on three downstream tasks (binary sentiment classification, topic categorization and natural language inference) with multilingual parallel test sets across 10 LRLs covering 6 language families in both unlabeled settings (+5.1%) and labeled settings (+16.3%). PARC-labeled also outperforms the finetuning baseline by 3.7%. We find a significant positive correlation between cross-lingual transfer performance on one side, and the similarity between the high- and low-resource languages as well as the amount of low-resource pretraining data on the other side. A robustness analysis suggests that PARC has the potential to achieve even stronger performance with more powerful MPLMs.
Code (1)
Tasks
Cross-Lingual TransferNatural Language InferenceRetrievalSentiment AnalysisSentiment ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Crosslingual Retrieval Augmented In-context Learning for Bangla
The promise of Large Language Models (LLMs) in Natural Language Processing has often been overshadowed by their limited performance in low-resource languages such as Bangla. To address this, our paper presents a pioneeri…
In-Context LearningRetrievalFrom Classification to Generation: Insights into Crosslingual Retrieval Augmented ICL
The remarkable ability of Large Language Models (LLMs) to understand and follow instructions has sometimes been limited by their in-context learning (ICL) performance in low-resource languages. To address this, we introd…
In-Context LearningRetrievalHow and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Prompting
Despite advances in the multilingual capabilities of Large Language Models (LLMs), their performance varies substantially across different languages and tasks. In multilingual retrieval-augmented generation (RAG)-based s…
Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Task
Retrieval-augmented generation (RAG) has become a cornerstone of contemporary NLP, enhancing large language models (LLMs) by allowing them to access richer factual contexts through in-context retrieval. While effective i…
Open-Domain Question AnsweringQuestion AnsweringRAGRetrieval+1Improving Retrieval-Augmented Neural Machine Translation with Monolingual Data
Conventional retrieval-augmented neural machine translation (RANMT) systems leverage bilingual corpora, e.g., translation memories (TMs). Yet, in many settings, in-domain monolingual target-side corpora are often availab…
Machine TranslationRetrievalSentenceTranslation