CoRoVA: Compressed Representations for Vector-Augmented Code Completion
Retrieval-augmented generation has emerged as one of the most effective approaches for code completion enhancement, especially when repository-level context is important. However, adding this extra retrieved context significantly increases sequence length, raises prefill cost, and degrades time-to-first-token (TTFT), which slows down inference -- a critical limitation for interactive settings such as IDEs. In this work, we introduce CoRoVA, a framework that compresses context into compact, semantically rich representations that remain interpretable to code LLMs. This improves generation quality while reducing prompt augmentation to only a few compressed single-token vectors. Our approach requires training only a small projector module and introduces negligible additional latency, yet it significantly improves the prediction quality of code LLMs. Our experiments show that CoRoVA enables a 20-38\% reduction in TTFT on completion tasks compared to uncompressed RAG.
Code (0)
등록된 구현이 없습니다.
Tasks
Code CompletionSimilar Papers 제목 키워드 기반
Augmented Deep Contexts for Spatially Embedded Video Coding
Most Neural Video Codecs (NVCs) only employ temporal references to generate temporal-only contexts and latent prior. These temporal-only NVCs fail to handle large motions or emerging objects due to limited contexts and m…
Learning representations for multivariate time series with missing data using Temporal Kernelized Autoencoders
Learning compressed representations of multivariate time series (MTS) facilitates data analysis in the presence of noise and redundant information, and for a large number of variates and time steps. However, classical di…
Dimensionality ReductionGeneral ClassificationImputationMissing Values+4Self-Supervised Learning of Compressed Video Representations
Self-supervised learning of video representations has recently received great attention. Existing methods typically require frames to be decoded before being processed, which increases compute and storage requirements an…
Computational EfficiencySelf-Supervised LearningDeCAL Tokenwise Compression
This paper introduces DeCAL, a new method for tokenwise compression. DeCAL uses an encoder-decoder language model pretrained with denoising to learn to produce high-quality, general-purpose compressed representations fro…
FaTRQ: Tiered Residual Quantization for LLM Vector Search in Far-Memory-Aware ANNS Systems
Approximate Nearest-Neighbor Search (ANNS) is a key technique in retrieval-augmented generation (RAG), enabling rapid identification of the most relevant high-dimensional embeddings from massive vector databases. Modern …