DecoupleSearch: Decouple Planning and Search via Hierarchical Reward Modeling

2025-09-07 · Hao Sun, Zile Qiao, Bo Wang, Guoxin Chen, Yingyan Hou, Yong Jiang, Pengjun Xie, Fei Huang, Yan Zhang arxiv

Retrieval-Augmented Generation (RAG) systems have emerged as a pivotal methodology for enhancing Large Language Models (LLMs) through the dynamic integration of external knowledge. To further improve RAG's flexibility, Agentic RAG introduces autonomous agents into the workflow. However, Agentic RAG faces several challenges: (1) the success of each step depends on both high-quality planning and accurate search, (2) the lack of supervision for intermediate reasoning steps, and (3) the exponentially large candidate space for planning and searching. To address these challenges, we propose DecoupleSearch, a novel framework that decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding. Our approach constructs a reasoning tree, where each node represents planning and search steps. We leverage Monte Carlo Tree Search to assess the quality of each step. During inference, Hierarchical Beam Search iteratively refines planning and search candidates with dual value models. Extensive experiments across policy models of varying parameter sizes demonstrate the effectiveness of our method.

📄 PDF Abstract BibTeX arXiv:2510.21712

Code (0)

등록된 구현이 없습니다.

Hierarchical Text Generation and Planning for Strategic Dialogue

2017-12-15 · ICML 2018 7 · Denis Yarats, Mike Lewis

End-to-end models for goal-orientated dialogue are challenging to train, because linguistic and strategic aspects are entangled in latent state vectors. We introduce an approach to learning representations of messages in…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+2

Strict Subgoal Execution: Reliable Long-Horizon Planning in Hierarchical Reinforcement Learning

2025-06-26 · Jaebak Hwang, Sanghyeon Lee, Jeongmo Kim, Seungyul Han

Long-horizon goal-conditioned tasks pose fundamental challenges for reinforcement learning (RL), particularly when goals are distant and rewards are sparse. While hierarchical and graph-based methods offer partial soluti…

Decision Making Hierarchical Reinforcement Learning Reinforcement Learning (RL)

HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving

2026-04-04 · Wenhao Yao, Xinglong Sun, Zhenxin Li, Shiyi Lan 외 arxiv

End-to-end planning has emerged as a dominant paradigm for autonomous driving, where recent models often adopt a scoring-selection framework to choose trajectories from a large set of candidates, with diffusion-based dec…

Reinforcement Learning Autonomous Driving

HiRA: A Hierarchical Reasoning Framework for Decoupled Planning and Execution in Deep Search

2025-07-03 · Jiajie Jin, Xiaoxi Li, Guanting Dong, Yuyao Zhang 외 arxiv

Complex information needs in real-world search scenarios demand deep reasoning and knowledge synthesis across diverse sources, which traditional retrieval-augmented generation (RAG) pipelines struggle to address effectiv…

Proposing Hierarchical Goal-Conditioned Policy Planning in Multi-Goal Reinforcement Learning

2025-01-03 · Gavin B. Rens

Humanoid robots must master numerous tasks with sparse rewards, posing a challenge for reinforcement learning (RL). We propose a method combining RL and automated planning to address this. Our approach uses short goal-co…

Multi-Goal Reinforcement Learning Reinforcement Learning (RL)