Synthetic Cross-language Information Retrieval Training Data
A key stumbling block for neural cross-language information retrieval (CLIR) systems has been the paucity of training data. The appearance of the MS MARCO monolingual training set led to significant advances in the state of the art in neural monolingual retrieval. By translating the MS MARCO documents into other languages using machine translation, this resource has been made useful to the CLIR community. Yet such translation suffers from a number of problems. While MS MARCO is a large resource, it is of fixed size; its genre and domain of discourse are fixed; and the translated documents are not written in the language of a native speaker of the language, but rather in translationese. To address these problems, we introduce the JH-POLO CLIR training set creation methodology. The approach begins by selecting a pair of non-English passages. A generative large language model is then used to produce an English query for which the first passage is relevant and the second passage is not relevant. By repeating this process, collections of arbitrary size can be created in the style of MS MARCO but using naturally-occurring documents in any desired genre and domain of discourse. This paper describes the methodology in detail, shows its use in creating new CLIR training sets, and describes experiments using the newly created training data.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalLanguage ModellingLarge Language ModelMachine TranslationRetrievalTranslationSimilar Papers 제목 키워드 기반
Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval
There has been limited success for dense retrieval models in multilingual retrieval, due to uneven and scarce training data available across multiple languages. Synthetic training data generation is promising (e.g., InPa…
Language ModelingLanguage ModellingLarge Language ModelRetrievalInPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrieval
Recently, InPars introduced a method to efficiently use large language models (LLMs) in information retrieval tasks: via few-shot examples, an LLM is induced to generate relevant queries for documents. These synthetic qu…
Information RetrievalRetrievalGenerating Synthetic Documents for Cross-Encoder Re-Rankers: A Comparative Study of ChatGPT and Human Experts
We investigate the usefulness of generative Large Language Models (LLMs) in generating training data for cross-encoder re-rankers in a novel direction: generating synthetic documents instead of synthetic queries. We intr…
Re-RankingRetrievalREANIMATOR: Reanimate Retrieval Test Collections with Extracted and Synthetic Resources
Retrieval test collections are essential for evaluating information retrieval systems, yet they often lack generalizability across tasks. To overcome this limitation, we introduce REANIMATOR, a versatile framework design…
Information RetrievalRetrievalRetrieval-augmented GenerationTowards Robust Neural Retrieval Models with Synthetic Pre-Training
Recent work has shown that commonly available machine reading comprehension (MRC) datasets can be used to train high-performance neural information retrieval (IR) systems. However, the evaluation of neural IR has so far …
Information RetrievalMachine Reading ComprehensionReading ComprehensionRetrieval