Knowledgeable-r1: Policy Optimization for Knowledge Exploration in Retrieval-Augmented Generation

2025-06-05 · Chenyu Lin, Yilin Wen, Du Su, Fei Sun, Muhan Chen, Chenfu Bao, Zhonghou Lv

Retrieval-augmented generation (RAG) is a mainstream method for improving performance on knowledge-intensive tasks. However,current RAG systems often place too much emphasis on retrieved contexts. This can lead to reliance on inaccurate sources and overlook the model's inherent knowledge, especially when dealing with misleading or excessive information. To resolve this imbalance, we propose Knowledgeable-r1 that using joint sampling and define multi policy distributions in knowledge capability exploration to stimulate large language models'self-integrated utilization of parametric and contextual knowledge. Experiments show that Knowledgeable-r1 significantly enhances robustness and reasoning accuracy in both parameters and contextual conflict tasks and general RAG tasks, especially outperforming baselines by 17.07% in counterfactual scenarios and demonstrating consistent gains across RAG tasks. Our code are available at https://github.com/lcy80366872/ knowledgeable-r1.

📄 PDF Abstract BibTeX arXiv:2506.05154

Code (1)

lcy80366872/knowledgeable-r1 공식 구현 pytorch

Tasks

counterfactualRAG RetrievalRetrieval-augmented Generation

Methods 이 논문이 사용한 방법론

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization

2026-03-03 · Siwei Zhang, Yun Xiong, Xi Chen, Zi'an Jia 외 arxiv

Agentic Reinforcement Learning (Agentic RL) has shown remarkable potential in large language model-based (LLM) agents. These works can empower LLM agents to tackle complex tasks via multi-step, tool-integrated reasoning.…

Reinforcement Learning

Dynamic Injection of Entity Knowledge into Dense Retrievers

2025-07-05 · Ikuya Yamada, Ryokan Ri, Takeshi Kojima, Yusuke Iwasawa 외 arxiv

Dense retrievers often struggle with queries involving less-frequent entities due to their limited entity knowledge. We propose the Knowledgeable Passage Retriever (KPR), a BERT-based retriever enhanced with a context-en…

Embedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models

2024-04-04 · Chengkai Huang, Yu Xia, Rui Wang, Kaige Xie 외

Retrieval-augmented large language models (LLMs) have been remarkably competent in various NLP tasks. However, it was observed by previous works that retrieval is not always helpful, especially when the LLM is already kn…

Continual Learning Retrieval Retrieval-augmented Generation

Pixel-Grounded Retrieval for Knowledgeable Large Multimodal Models

2026-01-27 · Jeonghwan Kim, Renjie Tao, Sanat Sharma, Jiaqi Wang 외 arxiv

Visual Question Answering (VQA) often requires coupling fine-grained perception with factual knowledge beyond the input image. Prior multimodal Retrieval-Augmented Generation (MM-RAG) systems improve factual grounding bu…

Visual Question Answering Image Retrieval

Hierarchical Neural Network for Extracting Knowledgeable Snippets and Documents

2018-08-22 · Ganbin Zhou, Rongyu Cao, Xiang Ao, Ping Luo 외

In this study, we focus on extracting knowledgeable snippets and annotating knowledgeable documents from Web corpus, consisting of the documents from social media and We-media. Informally, knowledgeable snippets refer to…

Knowledge Base Construction