LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval
In this paper, we propose LaPraDoR, a pretrained dual-tower dense retriever that does not require any supervised data for training. Specifically, we first present Iterative Contrastive Learning (ICoL) that iteratively trains the query and document encoders with a cache mechanism. ICoL not only enlarges the number of negative instances but also keeps representations of cached examples in the same hidden space. We then propose Lexicon-Enhanced Dense Retrieval (LEDR) as a simple yet effective way to enhance dense retrieval with lexical matching. We evaluate LaPraDoR on the recently proposed BEIR benchmark, including 18 datasets of 9 zero-shot text retrieval tasks. Experimental results show that LaPraDoR achieves state-of-the-art performance compared with supervised dense retrieval models, and further analysis reveals the effectiveness of our training strategy and objectives. Compared to re-ranking, our lexicon-enhanced approach can be run in milliseconds (22.5x faster) while achieving superior performance.
Code (1)
Tasks
Contrastive LearningRe-RankingRetrievalText RetrievalZero-shot Text RetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Leveraging LLMs for Unsupervised Dense Retriever Ranking
In this paper we present Large Language Model Assisted Retrieval Model Ranking (LARMOR), an effective unsupervised approach that leverages LLMs for selecting which dense retriever to use on a test corpus (target). Dense …
Language ModellingLarge Language ModelBoot and Switch: Alternating Distillation for Zero-Shot Dense Retrieval
Neural 'dense' retrieval models are state of the art for many datasets, however these models often exhibit limited domain transfer ability. Existing approaches to adaptation are unwieldy, such as requiring explicit super…
Passage RetrievalRetrievalPrecise Zero-Shot Dense Retrieval without Relevance Labels
While dense retrieval has been shown effective and efficient across tasks and languages, it remains difficult to create effective fully zero-shot dense retrieval systems when no relevance label is available. In this pape…
Fact VerificationInstruction FollowingLanguage ModelingLanguage Modelling+2Selecting which Dense Retriever to use for Zero-Shot Search
We propose the new problem of choosing which dense retrieval model to use when searching on a new collection for which no labels are available, i.e. in a zero-shot setting. Many dense retrieval models are readily availab…
Information RetrievalRetrievalCan a Frozen Pretrained Language Model be used for Zero-shot Neural Retrieval on Entity-centric Questions?
Neural document retrievers, including dense passage retrieval (DPR), have outperformed classical lexical-matching retrievers, such as BM25, when fine-tuned and tested on specific question-answering datasets. However, it …
Domain GeneralizationLanguage ModelingLanguage ModellingPassage Retrieval+2