Learning Cross-Lingual IR from an English Retriever
We present DR.DECR (Dense Retrieval with Distillation-Enhanced Cross-Lingual Representation), a new cross-lingual information retrieval (CLIR) system trained using multi-stage knowledge distillation (KD). The teacher of DR.DECR relies on a highly effective but computationally expensive two-stage inference process consisting of query translation and monolingual IR, while the student, DR.DECR, executes a single CLIR step. We teach DR.DECR powerful multilingual representations as well as CLIR by optimizing two corresponding KD objectives. Learning useful representations of non-English text from an English-only retriever is accomplished through a cross-lingual token alignment algorithm that relies on the representation capabilities of the underlying multilingual encoders. In both in-domain and zero-shot out-of-domain evaluation, DR.DECR demonstrates far superior accuracy over direct fine-tuning with labeled CLIR data. It is also the best single-model retriever on the XOR-TyDi benchmark at the time of this writing.
Code (1)
Tasks
Cross-Lingual Information RetrievalInformation RetrievalKnowledge DistillationMachine TranslationRetrievalTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples
Recent studies indicate that leveraging off-the-shelf or fine-tuned retrievers, capable of retrieving relevant in-context examples tailored to the input query, enhances few-shot in-context learning of English. However, a…
In-Context LearningLanguage ModelingLanguage ModellingLarge Language Model+5Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training
Keyphrase generation is the task of automatically predicting keyphrases given a piece of long text. Despite its recent flourishing, keyphrase generation on non-English languages haven't been vastly investigated. In this …
Keyphrase GenerationPassage RetrievalRetrievalLearning Cross-Lingual IR from an English Retriever
We present a new cross-lingual information retrieval (CLIR) system trained using multi-stage knowledge distillation (KD). The teacher relies on a highly effective but expensive two-stage process consisting of query trans…
Cross-Lingual Information RetrievalInformation RetrievalKnowledge DistillationLanguage Modeling+3CLIRudit: Cross-Lingual Information Retrieval of Scientific Documents
Cross-lingual information retrieval (CLIR) consists in finding relevant documents in a language that differs from the language of the queries. This paper presents CLIRudit, a new dataset created to evaluate cross-lingual…
BenchmarkingCross-Lingual Information RetrievalDocument TranslationInformation Retrieval+3mFollowIR: a Multilingual Benchmark for Instruction Following in Retrieval
Retrieval systems generally focus on web-style queries that are short and underspecified. However, advances in language models have facilitated the nascent rise of retrieval models that can understand more complex querie…
Instruction FollowingRetrieval