Reranking with Compressed Document Representation

2025-05-21 · Hervé Déjean, Stéphane Clinchant

Reranking, the process of refining the output of a first-stage retriever, is often considered computationally expensive, especially with Large Language Models. Borrowing from recent advances in document compression for RAG, we reduce the input size by compressing documents into fixed-size embedding representations. We then teach a reranker to use compressed inputs by distillation. Although based on a billion-size model, our trained reranker using this compressed input can challenge smaller rerankers in terms of both effectiveness and efficiency, especially for long documents. Given that text compressors are still in their early development stages, we view this approach as promising.

📄 PDF Abstract BibTeX arXiv:2505.15394

Code (0)

등록된 구현이 없습니다.

Tasks

RAGReranking

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

WordPiece 설명 없음

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Residual Connection 설명 없음

SDR: Efficient Neural Re-ranking using Succinct Document Representation

2021-10-03 · ACL 2022 5 · Nachshon Cohen, Amit Portnoy, Besnik Fetahu, Amir Ingber

BERT based ranking models have achieved superior performance on various information retrieval tasks. However, the large number of parameters and complex self-attention operation come at a significant latency overhead. To…

Information Retrieval Quantization Reranking Re-Ranking+1

LLM-Based Compact Reranking with Document Features for Scientific Retrieval

2025-05-19 · Runchu Tian, Xueqiang Xu, Bowen Jin, SeongKu Kang 외

Scientific retrieval is essential for advancing academic discovery. Within this process, document reranking plays a critical role by refining first-stage retrieval results. However, large language model (LLM) listwise re…

Large Language Model Reranking Retrieval

KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking

2026-06-22 · Xinping Zhao, Jiaxin Xu, Ziqi Dai, Xin Zhang 외 arxiv

As retrieval systems scale, high-quality reranking becomes increasingly important. However, most existing rerankers, whether encoder-based or decoder-based, jointly encode the query and passage, tightly coupling their co…

Knowledge-Aided Open-Domain Question Answering

2020-06-09 · Mantong Zhou, Zhouxing Shi, Minlie Huang, Xiaoyan Zhu

Open-domain question answering (QA) aims to find the answer to a question from a large collection of documents.Though many models for single-document machine comprehension have achieved strong performance, there is still…

Open-Domain Question Answering Question Answering Reading Comprehension Reranking+1

DiffuRank: Effective Document Reranking with Diffusion Language Models

2026-02-13 · Qi Liu, Kun Ai, Jiaxin Mao, Yanzhao Zhang 외 arxiv

Recent advances in large language models (LLMs) have inspired new paradigms for document reranking. While this paradigm better exploits the reasoning and contextual understanding capabilities of LLMs, most existing LLM-b…