Instance-based Transfer Learning for Multilingual Deep Retrieval
We focus on the problem of search in the multilingual setting. Examining the problems of next-sentence prediction and inverse cloze, we show that at large scale, instance-based transfer learning is surprisingly effective in the multilingual setting, leading to positive transfer on all of the 35 target languages and two tasks tested. We analyze this improvement and argue that the most natural explanation, namely direct vocabulary overlap between languages, only partially explains the performance gains: in fact, we demonstrate target-language improvement can occur after adding data from an auxiliary language even with no vocabulary in common with the target. This surprising result is due to the effect of transitive vocabulary overlaps between pairs of auxiliary and target languages.
Code (0)
등록된 구현이 없습니다.
Tasks
RetrievalSentenceTransfer LearningSimilar Papers 제목 키워드 기반
Soft Language Clustering for Multilingual Model Pre-training
Multilingual pre-trained language models have demonstrated impressive (zero-shot) cross-lingual transfer abilities, however, their performance is hindered when the target language has distant typology from source languag…
ClusteringCross-Lingual TransfermodelQuestion Answering+6Parameter-efficient Zero-shot Transfer for Cross-Language Dense Retrieval with Adapters
A popular approach to creating a zero-shot cross-language retrieval model is to substitute a monolingual pretrained language model in the retrieval model with a multilingual pretrained language model such as Multilingual…
Information RetrievalLanguage ModelingLanguage ModellingRetrievalOn Cross-Lingual Retrieval with Multilingual Text Encoders
In this work we present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks across a number of diverse language pa…
Cross-Lingual TransferRe-RankingRetrievalSentence+2Data-Efficient Hate Speech Detection via Cross-Lingual Nearest Neighbor Retrieval with Limited Labeled Data
Considering the importance of detecting hateful language, labeled hate speech data is expensive and time-consuming to collect, particularly for low-resource languages. Prior work has demonstrated the effectiveness of cro…
Cross-Lingual TransferData AugmentationHate Speech DetectionRetrieval+1Soft Prompt Decoding for Multilingual Dense Retrieval
In this work, we explore a Multilingual Information Retrieval (MLIR) task, where the collection includes documents in multiple languages. We demonstrate that applying state-of-the-art approaches developed for cross-lingu…
Cross-Lingual Information RetrievalInformation RetrievalKnowledge DistillationRetrieval