Operational Advice for Dense and Sparse Retrievers: HNSW, Flat, or Inverted Indexes?
Practitioners working on dense retrieval today face a bewildering number of choices. Beyond selecting the embedding model, another consequential choice is the actual implementation of nearest-neighbor vector search. While best practices recommend HNSW indexes, flat vector indexes with brute-force search represent another viable option, particularly for smaller corpora and for rapid prototyping. In this paper, we provide experimental results on the BEIR dataset using the open-source Lucene search library that explicate the tradeoffs between HNSW and flat indexes (including quantized variants) from the perspectives of indexing time, query evaluation performance, and retrieval quality. With additional comparisons between dense and sparse retrievers, our results provide guidance for today's search practitioner in understanding the design space of dense and sparse retrievers. To our knowledge, we are the first to provide operational advice supported by empirical experiments in this regard.
Code (0)
등록된 구현이 없습니다.
Tasks
RetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Anserini Gets Dense Retrieval: Integration of Lucene's HNSW Indexes
Anserini is a Lucene-based toolkit for reproducible information retrieval research in Java that has been gaining traction in the community. It provides retrieval capabilities for both "traditional" bag-of-words retrieval…
Information RetrievalRetrievalPredicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selection
Over the last few years, contextualized pre-trained transformer models such as BERT have provided substantial improvements on information retrieval tasks. Recent approaches based on pre-trained transformer models such as…
GPUInformation RetrievalRetrievalTo Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers
Current pre-trained language model approaches to information retrieval can be broadly divided into two categories: sparse retrievers (to which belong also non-neural approaches such as bag-of-words methods, e.g., BM25) a…
Information RetrievalLanguage ModellingRetrievalA Study on the Efficiency and Generalization of Light Hybrid Retrievers
Hybrid retrievers can take advantage of both sparse and dense retrievers. Previous hybrid retrievers leverage indexing-heavy dense retrievers. In this work, we study "Is it possible to reduce the indexing memory of hybri…
Adversarial AttackContrastive LearningKnowledge DistillationSalient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?
Despite their recent popularity and well known advantages, dense retrievers still lag behind sparse methods such as BM25 in their ability to reliably match salient phrases and rare entities in the query. It has been argu…
Open-Domain Question AnsweringPassage RetrievalQuestion AnsweringRetrieval