홈 › Papers

ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation

2026-01-31 · Shihao Wang, Jiahao Chen, Yanqi Pan, Hao Huang, Yichen Hao, Xiangyu Zou, Wen Xia, Wentao Zhang, Chongyang Qiu, Pengfei Wang arxiv

The prefill stage of long-context Retrieval-Augmented Generation (RAG) is severely bottlenecked by computational overhead. To mitigate this, recent methods assemble pre-calculated KV caches of retrieved RAG documents (by a user query) and reprocess selected tokens to recover cross-attention between these pre-calculated KV caches. However, we identify a fundamental "crowding-out effect" in current token selection criteria: globally salient but user-query-irrelevant tokens saturate the limited recomputation budget, displacing the tokens truly essential for answering the user query and degrading inference accuracy. We propose ProphetKV, a user-query-driven KV Cache reuse method for RAG scenarios. ProphetKV dynamically prioritizes tokens based on their semantic relevance to the user query and employs a dual-stage recomputation pipeline to fuse layer-wise attention metrics into a high-utility set. By ensuring the recomputation budget is dedicated to bridging the informational gap between retrieved context and the user query, ProphetKV achieves high-fidelity attention recovery with minimal overhead. Our extensive evaluation results show that ProphetKV retains 96%-101% of full-prefill accuracy with only a 20% recomputation ratio, while achieving accuracy improvements of 8.8%-24.9% on RULER and 18.6%-50.9% on LongBench over the state-of-the-art approaches (e.g., CacheBlend, EPIC, and KVShare).

📄 PDF Abstract BibTeX arXiv:2602.02579

Code (0)

등록된 구현이 없습니다.

QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving

2026-06-04 · Jianxin Yan, Wangze Ni, Zhenxin Li, Jiabao Jin 외 arxiv

Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing retrieved contexts makes the prefill stage a dominant serving cost. RAG…

QCFuse: Query-Centric Cache Fusion for Efficient RAG Inference

2026-03-30 · Jianxin Yan, Zeheng Qian, Wangze Ni, Zhitao Shen 외 arxiv

Cache fusion accelerates generation process of LLMs equipped with RAG through KV caching and selective token recomputation, thereby reducing computational costs and improving efficiency. However, existing methods primari…

InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context

2026-03-05 · Xin Teng, Canyu Zhang, Shaoyi Zheng, Danyang Zhuo 외 arxiv

Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precompute key-value (KV) caches for individual…

Question Answering

SIFT: Selective-Index For Fast Compute of RAG Prefill by Exploiting Attention Invariance

2026-06-08 · Rya Sanovar, Srikant Bharadwaj, Hritvik Taneja, Moinuddin Qureshi arxiv

Retrieval-Augmented Generation (RAG) injects LLM queries with relevant documents to improve response quality. This injection increases prompt length and slows time to first token (TTFT). Unlike standard queries, RAG quer…

A Graph Theoretic Framework of Recomputation Algorithms for Memory-Efficient Backpropagation

2019-05-28 · NeurIPS 2019 12 · Mitsuru Kusumoto, Takuya Inoue, Gentaro Watanabe, Takuya Akiba 외

Recomputation algorithms collectively refer to a family of methods that aims to reduce the memory consumption of the backpropagation by selectively discarding the intermediate results of the forward propagation and recom…