XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples
Recent studies indicate that leveraging off-the-shelf or fine-tuned retrievers, capable of retrieving relevant in-context examples tailored to the input query, enhances few-shot in-context learning of English. However, adapting these methods to other languages, especially low-resource ones, poses challenges due to the scarcity of cross-lingual retrievers and annotated data. Thus, we introduce XAMPLER: Cross-Lingual Example Retrieval, a method tailored to tackle the challenge of cross-lingual in-context learning using only annotated English data. XAMPLER first trains a retriever based on Glot500, a multilingual small language model, using positive and negative English examples constructed from the predictions of a multilingual large language model, i.e., MaLA500. Leveraging the cross-lingual capacity of the retriever, it can directly retrieve English examples as few-shot examples for in-context learning of target languages. Experiments on the multilingual text classification benchmark SIB200 with 176 languages show that XAMPLER substantially improves the in-context learning performance across languages. Our code is available at \url{https://github.com/cisnlp/XAMPLER}.
Code (1)
Tasks
In-Context LearningLanguage ModelingLanguage ModellingLarge Language ModelMultilingual text classificationRetrievalSmall Language Modeltext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Examplers based image fusion features for face recognition
Examplers of a face are formed from multiple gallery images of a person and are used in the process of classification of a test image. We incorporate such examplers in forming a biologically inspired local binary decisio…
Face RecognitionGeneral ClassificationAlign, Generate, Learn: A Novel Closed-Loop Framework for Cross-Lingual In-Context Learning
Cross-lingual in-context learning (XICL) has emerged as a transformative paradigm for leveraging large language models (LLMs) to tackle multilingual tasks, especially for low-resource languages. However, existing approac…
In-Context LearningCONCAP: Seeing Beyond English with Concepts Retrieval-Augmented Captioning
Multilingual vision-language models have made significant strides in image captioning, yet they still lag behind their English counterparts due to limited multilingual training data and costly large-scale model parameter…
Image CaptioningCross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance
Multilingual large language models (MLLMs) have demonstrated significant cross-lingual capabilities through in-context learning. Existing approaches typically construct monolingual in-context examples, either in the sour…
Cross-Lingual TransferIn-Context LearningQuestion AnsweringXRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing
In-context learning using large language models has recently shown surprising results for semantic parsing tasks such as Text-to-SQL translation. Prompting GPT-3 or Codex using several examples of question-SQL pairs can …
In-Context LearningRetrievalSemantic ParsingText to SQL+3