Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning

2025-02-20 · Jiachen Zhu, Congmin Zheng, Jianghao Lin, Kounianhua Du, Ying Wen, Yong Yu, Jun Wang, Weinan Zhang

While large language models (LLMs) have significantly advanced mathematical reasoning, Process Reward Models (PRMs) have been developed to evaluate the logical validity of reasoning steps. However, PRMs still struggle with out-of-distribution (OOD) challenges. This paper identifies key OOD issues, including step OOD, caused by differences in reasoning patterns across model types and sizes, and question OOD, which arises from dataset shifts between training data and real-world problems. To address these issues, we introduce Retrieval-Augmented Process Reward Model (RetrievalPRM), a novel framework designed to tackle these OOD issues. By utilizing a two-stage retrieval-enhanced mechanism, RetrievalPRM retrieves semantically similar questions and steps as a warmup, enhancing PRM's ability to evaluate target steps and improving generalization and reasoning consistency across different models and problem types. Our extensive experiments demonstrate that RetrievalPRM outperforms existing baselines across multiple real-world datasets. Our open-source contributions include a retrieval-enhanced dataset, a tuning framework for PRM training, and the RetrievalPRM model, establishing a new standard for PRM performance.

📄 PDF Abstract BibTeX arXiv:2502.14361

Code (0)

등록된 구현이 없습니다.

Tasks

Mathematical Reasoning Retrieval

KG-Augmented Executable CoT for Mathematical Coding

2025-08-06 · Xingyu Chen, Junxiu An, Jun Guo, Li Wang 외 arxiv

In recent years, large language models (LLMs) have excelled in natural language processing tasks but face significant challenges in complex reasoning tasks such as mathematical reasoning and code generation. To address t…

Mathematical Reasoning Knowledge Graphs Code Generation

RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation

2025-01-23 · Shi-Qi Yan, Zhen-Hua Ling

While Retrieval-Augmented Generation (RAG) has exhibited promise in utilizing external knowledge, its generation process heavily depends on the quality and accuracy of the retrieved context. Large language models (LLMs) …

Memorization RAG Response Generation Retrieval+1

Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning

2026-06-11 · Zilin Xiao, Qi Ma, Chun-cheng Jason Chen, Xintao Chen 외 arxiv

Retrieval-augmented generation (RAG) has become a standard mechanism for grounding language models in external knowledge, yet conventional retrieval based on lexical or semantic similarity is poorly suited for complex re…

Mathematical Reasoning Semantic Similarity

Retrieval Augmented Generation-Enhanced Distributed LLM Agents for Generalizable Traffic Signal Control with Emergency Vehicles

2025-10-30 · Xinhang Li, Qing Guo, Junyu Chen, Zheng Guo 외 arxiv

With increasing urban traffic complexity, Traffic Signal Control (TSC) is essential for optimizing traffic flow and improving road safety. Large Language Models (LLMs) emerge as promising approaches for TSC. However, the…

ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding

2025-01-14 · Zhongxiang Sun, QiPeng Wang, Weijie Yu, Xiaoxue Zang 외

Retrieval-Augmented Generation (RAG) systems for Large Language Models (LLMs) hold promise in knowledge-intensive tasks but face limitations in complex multi-step reasoning. While recent methods have integrated RAG with …

RAG Retrieval Retrieval-augmented Generation