ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation

2025-02-12 · Ruobing Yao, Yifei Zhang, Shuang Song, Yuhua Liu, Neng Gao, Chenyang Tu

While Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by incorporating external knowledge, they still face persistent challenges in retrieval inefficiency and the inability of LLMs to filter out irrelevant information. We present ParetoRAG, an unsupervised framework that optimizes RAG systems through sentence-level refinement guided by the Pareto principle. By decomposing paragraphs into sentences and dynamically re-weighting core content while preserving contextual coherence, ParetoRAG achieves dual improvements in both retrieval precision and generation quality without requiring additional training or API resources. This framework has been empirically validated across various datasets, LLMs, and retrievers.

📄 PDF Abstract BibTeX arXiv:2502.08178

Code (0)

등록된 구현이 없습니다.

Tasks

RAG RetrievalRetrieval-augmented GenerationSentence

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Attention 설명 없음

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

You Only Use Reactive Attention Slice For Long Context Retrieval

2024-09-03 · Yun Joon Soh, Hanxian Huang, Yuandong Tian, Jishen Zhao

Supporting longer context for Large Language Models (LLM) is a promising direction to advance LLMs. As training a model for a longer context window is computationally expensive, many alternative solutions, such as Retrie…

RAG Retrieval Retrieval-augmented Generation Sentence

SproutRAG: Attention-Guided Tree Search with Progressive Embeddings for Long-Document RAG

2026-06-16 · Amirhossein Abaskohi, Issam H. Laradji, Peter West, Giuseppe Carenini arxiv

Retrieval-augmented generation (RAG) systems must balance retrieval granularity with contextual coherence, a challenge that existing methods address through LLM-guided chunking, single-level context expansion, or hierarc…

Cross-modal Semantic Enhanced Interaction for Image-Sentence Retrieval

2022-10-17 · Xuri Ge, Fuhai Chen, Songpei Xu, Fuxiang Tao 외

Image-sentence retrieval has attracted extensive research attention in multimedia and computer vision due to its promising application. The key issue lies in jointly learning the visual and textual representation to accu…

cross-modal alignment Object Retrieval Sentence+1

Say Less, Mean More: Leveraging Pragmatics in Retrieval-Augmented Generation

2025-02-25 · Haris Riaz, Ellen Riloff, Mihai Surdeanu

We propose a simple, unsupervised method that injects pragmatic principles in retrieval-augmented generation (RAG) frameworks such as Dense Passage Retrieval~\cite{karpukhin2020densepassageretrievalopendomain} to enhance…

ARC Passage Retrieval Question Answering RAG+2

What Makes Good In-Context Examples for GPT-3?

2022-05-01 · DeeLIO (ACL) 2022 5 · Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan 외

GPT-3 has attracted lots of attention due to its superior performance across a wide range of NLP tasks, especially with its in-context learning abilities. Despite its success, we found that the empirical results of GPT-3…

In-Context Learning Natural Language Understanding Open-Domain Question Answering Question Answering+4