Equipping Retrieval-Augmented Large Language Models with Document Structure Awareness

2025-10-05 · Lingnan Xu, Chong Feng, Kaiyuan Zhang, Liu Zhengyong, Wenqiang Xu, Fanqing Meng arxiv

While large language models (LLMs) demonstrate impressive capabilities, their reliance on parametric knowledge often leads to factual inaccuracies. Retrieval-Augmented Generation (RAG) mitigates this by leveraging external documents, yet existing approaches treat retrieved passages as isolated chunks, ignoring valuable structure that is crucial for document organization. Motivated by this gap, we propose Retrieve-DocumentRoute-Read (RDR2), a novel framework that explicitly incorporates structural information throughout the RAG process. RDR2 employs an LLM-based router to dynamically navigate document structure trees, jointly evaluating content relevance and hierarchical relationships to assemble optimal evidence. Our key innovation lies in formulating document routing as a trainable task, with automatic action curation and structure-aware passage selection inspired by human reading strategies. Through comprehensive evaluation on five challenging datasets, RDR2 achieves state-of-the-art performance, demonstrating that explicit structural awareness significantly enhances RAG systems' ability to acquire and utilize knowledge, particularly in complex scenarios requiring multi-document synthesis.

📄 PDF Abstract BibTeX arXiv:2510.04293

Code (0)

등록된 구현이 없습니다.

RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

2024-03-31 · Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo 외

Large Language Models (LLMs) exhibit remarkable capabilities but are prone to generating inaccurate or hallucinatory responses. This limitation stems from their reliance on vast pretraining datasets, making them suscepti…

In-Context Learning RAG Response Generation Retrieval+1

A Large Language Model-based Framework for Semi-Structured Tender Document Retrieval-Augmented Generation

2024-10-04 · Yilong Zhao, Daifeng Li

The drafting of documents in the procurement field has progressively become more complex and diverse, driven by the need to meet legal requirements, adapt to technological advancements, and address stakeholder demands. W…

Language Modeling Language Modelling Large Language Model Retrieval+1

Building A Coding Assistant via the Retrieval-Augmented Language Model

2024-10-21 · Xinze Li, Hanbin Wang, Zhenghao Liu, Shi Yu 외

Pretrained language models have shown strong effectiveness in code-related tasks, such as code retrieval, code generation, code summarization, and code completion tasks. In this paper, we propose COde assistaNt viA retri…

Code Completion Code Generation Code Summarization Language Modeling+3

Retrieval-Augmented Machine Translation with Unstructured Knowledge

2024-12-05 · Jiaan Wang, Fandong Meng, Yingxue Zhang, Jie zhou

Retrieval-augmented generation (RAG) introduces additional information to enhance large language models (LLMs). In machine translation (MT), previous work typically retrieves in-context examples from paired MT corpora, o…

Knowledge Graphs Machine Translation RAG Retrieval+3

KG-Retriever: Efficient Knowledge Indexing for Retrieval-Augmented Large Language Models

2024-12-07 · WeiJie Chen, Ting Bai, Jinbo Su, Jian Luan 외

Large language models with retrieval-augmented generation encounter a pivotal challenge in intricate retrieval tasks, e.g., multi-hop question answering, which requires the model to navigate across multiple documents and…

Multi-hop Question Answering Navigate Question Answering RAG+2