Optimizing Test-Time Query Representations for Dense Retrieval
Recent developments of dense retrieval rely on quality representations of queries and contexts from pre-trained query and context encoders. In this paper, we introduce TOUR (Test-Time Optimization of Query Representations), which further optimizes instance-level query representations guided by signals from test-time retrieval results. We leverage a cross-encoder re-ranker to provide fine-grained pseudo labels over retrieval results and iteratively optimize query representations with gradient descent. Our theoretical analysis reveals that TOUR can be viewed as a generalization of the classical Rocchio algorithm for pseudo relevance feedback, and we present two variants that leverage pseudo-labels as hard binary or soft continuous labels. We first apply TOUR on phrase retrieval with our proposed phrase re-ranker, and also evaluate its effectiveness on passage retrieval with an off-the-shelf re-ranker. TOUR greatly improves end-to-end open-domain question answering accuracy, as well as passage retrieval performance. TOUR also consistently improves direct re-ranking by up to 2.0% while running 1.3-2.4x faster with an efficient implementation.
Code (1)
Tasks
Contrastive LearningInformation RetrievalOpen-Domain Question AnsweringPassage RetrievalQuestion AnsweringRe-RankingRetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Contextualized Query Embeddings for Conversational Search
This paper describes a compact and effective model for low-latency passage retrieval in conversational search based on learned dense representations. Prior to our work, the state-of-the-art approach uses a multi-stage pi…
Conversational SearchInformation RetrievalOpen-Domain Question AnsweringPassage Retrieval+2Augmenting Passage Representations with Query Generation for Enhanced Cross-Lingual Dense Retrieval
Effective cross-lingual dense retrieval methods that rely on multilingual pre-trained language models (PLMs) need to be trained to encompass both the relevance matching task and the cross-language alignment task. However…
Cross-Lingual Information RetrievalInformation RetrievalRetrievalLearning to Select: Query-Aware Adaptive Dimension Selection for Dense Retrieval
Dense retrieval represents queries and documents as high-dimensional embeddings, but these representations can be redundant at the query level: for a given information need, only a subset of dimensions is consistently he…
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Dense retrieval systems conduct first-stage retrieval using embedded representations and simple similarity metrics to match a query to documents. Its effectiveness depends on encoded embeddings to capture the semantics o…
RetrievalCAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion
The dual-encoder has become the de facto architecture for dense retrieval. Typically, it computes the latent representations of the query and document independently, thus failing to fully capture the interactions between…
Passage RetrievalRetrieval