홈 › Papers

Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learning

2025-12-03 · Franki Nguimatsia Tiofack, Théotime Le Hellard, Fabian Schramm, Nicolas Perrin-Gilbert, Justin Carpentier arxiv

Offline reinforcement learning often relies on behavior regularization that enforces policies to remain close to the dataset distribution. However, such approaches fail to distinguish between high-value and low-value actions in their regularization components. We introduce Guided Flow Policy (GFP), which couples a multi-step flow-matching policy with a distilled one-step actor. The actor directs the flow policy through weighted behavior cloning to focus on cloning high-value actions from the dataset rather than indiscriminately imitating all state-action pairs. In turn, the flow policy constrains the actor to remain aligned with the dataset's best transitions while maximizing the critic. This mutual guidance enables GFP to achieve state-of-the-art performance across 144 state and pixel-based tasks from the OGBench, Minari, and D4RL benchmarks, with substantial gains on suboptimal datasets and challenging tasks. Webpage: https://simple-robotics.github.io/publications/guided-flow-policy/

📄 PDF Abstract BibTeX arXiv:2512.03973

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Potential-Guided Flow Matching for Vision-Language-Action Policy Improvement

2026-06-03 · Yunpeng Mei, Jiakai He, Hongjie Cao, Chenyu Wang 외 arxiv

Large vision-language-action (VLA) policies are increasingly trained as conditional generative models over action chunks. Yet deployment produces mixed-quality experience-successful demonstrations, partial completions, r…

Reinforcement Learning

Scalable Exploration for High-Dimensional Continuous Control via Value-Guided Flow

2026-01-27 · Yunyue Wei, Chenhui Zuo, Yanan Sui arxiv

Controlling high-dimensional systems in biological and robotic applications is challenging due to expansive state-action spaces, where effective exploration is critical. Commonly used exploration strategies in reinforcem…

Dimensionality Reduction Reinforcement Learning Continuous Control

Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning

2026-06-09 · Zhiyuan Zhou, Andy Peng, Charles Xu, Qiyang Li 외 arxiv

Expressive continuous control policies, such as diffusion and flow models, form the backbone of recent advances in scaling imitation learning for simulated and real robot control. While they are known to scale stably in …

Reinforcement Learning Continuous Control Offline RL

Q-Distribution guided Q-learning for offline reinforcement learning: Uncertainty penalized Q-value via consistency model

2024-10-27 · Jing Zhang, Linjiajie Fang, Kexin Shi, Wenjia Wang 외

``Distribution shift'' is the main obstacle to the success of offline reinforcement learning. A learning policy may take actions beyond the behavior policy's knowledge, referred to as Out-of-Distribution (OOD) actions. T…

D4RL Q-Learning

Sampling Efficient Deep Reinforcement Learning through Preference-Guided Stochastic Exploration

2022-06-20 · Wenhui Huang, Cong Zhang, Jingda Wu, Xiangkun He 외

Massive practical works addressed by Deep Q-network (DQN) algorithm have indicated that stochastic policy, despite its simplicity, is the most frequently used exploration approach. However, most existing stochastic explo…

Atari Games Deep Reinforcement Learning Q-Learning reinforcement-learning+2