paper-with-me

홈 › Papers

BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generation

2025-08-07 · Yuhao Wang, Ruiyang Ren, Yucheng Wang, Jing Liu, Wayne Xin Zhao, Hua Wu, Haifeng Wang arxiv

With the rapid advancement of large language models (LLMs), retrieval-augmented generation (RAG) has emerged as a critical approach to supplement the inherent knowledge limitations of LLMs. However, due to the typically large volume of retrieved information, RAG tends to operate with long context lengths. From the perspective of entropy engineering, we identify unconstrained entropy growth and attention dilution due to long retrieval context as significant factors affecting RAG performance. In this paper, we propose the balanced entropy-engineered RAG (BEE-RAG) framework, which improves the adaptability of RAG systems to varying context lengths through the principle of entropy invariance. By leveraging balanced context entropy to reformulate attention dynamics, BEE-RAG separates attention sensitivity from context length, ensuring a stable entropy level. Building upon this, we introduce a zero-shot inference strategy for multi-importance estimation and a parameter-efficient adaptive fine-tuning mechanism to obtain the optimal balancing factor for different settings. Extensive experiments across multiple RAG tasks demonstrate the effectiveness of BEE-RAG.

📄 PDF Abstract BibTeX arXiv:2508.05100

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

SF-RAG: Structure-Fidelity Retrieval-Augmented Generation for Academic Question Answering

2026-02-14 · Rui Yu, Tianyi Wang, Ruixia Liu, Yinglong Wang arxiv

Efficient question-answering (QA) over extensive scientific literature is essential for evidence-based engineering decision-making. Retrieval-augmented generation (RAG) is increasingly applied to question-answering over …

Question Answering

L-RAG: Balancing Context and Retrieval with Entropy-Based Lazy Loading

2026-01-10 · Sergii Voloshyn arxiv

Retrieval-Augmented Generation (RAG) has emerged as the predominant paradigm for grounding Large Language Model outputs in factual knowledge, effectively mitigating hallucinations. However, conventional RAG systems opera…

Network-informed Prompt Engineering against Organized Astroturf Campaigns under Extreme Class Imbalance

2025-01-21 · Nikos Kanakaris, Heng Ping, Xiongye Xiao, Nesreen K. Ahmed 외

Detecting organized political campaigns is of paramount importance in fighting against disinformation on social media. Existing approaches for the identification of such organized actions employ techniques mostly from ne…

Data AugmentationLanguage ModelingLanguage ModellingPrompt Engineering+2

Retrieval-Augmented Instruction Tuning for Automated Process Engineering Calculations : A Tool-Chaining Problem-Solving Framework with Attributable Reflection

2024-08-28 · Sagar Srinivas Sakhinana, Geethan Sannidhi, Venkataramana Runkana

The current technology landscape lacks a foundational AI model for solving process engineering calculations. In this work, we introduce a novel autonomous agent framework leveraging Retrieval-Augmented Instruction-Tuning…

Code Generationknowledge editingRetrieval

Rate-Distortion Limits for Multimodal Retrieval: Theory, Optimal Codes, and Finite-Sample Guarantees

2025-09-14 · Thomas Y. Chen arxiv

We establish the first information-theoretic limits for multimodal retrieval. Casting ranking as lossy source coding, we derive a single-letter rate-distortion function $R(D)$ for reciprocal-rank distortion and prove a c…