Partially Observable Reference Policy Programming: Solving POMDPs Sans Numerical Optimisation

2025-07-16 · Edward Kim, Hanna Kurniawati arxiv

This paper proposes Partially Observable Reference Policy Programming, a novel anytime online approximate POMDP solver which samples meaningful future histories very deeply while simultaneously forcing a gradual policy update. We provide theoretical guarantees for the algorithm's underlying scheme which say that the performance loss is bounded by the average of the sampling approximation errors rather than the usual maximum, a crucial requirement given the sampling sparsity of online planning. Empirical evaluations on two large-scale problems with dynamically evolving environments -- including a helicopter emergency scenario in the Corsica region requiring approximately 150 planning steps -- corroborate the theoretical results and indicate that our solver considerably outperforms current online benchmarks.

📄 PDF Abstract BibTeX arXiv:2507.12186

Code (0)

등록된 구현이 없습니다.

Reference-Based POMDPs

2023-09-21 · NeurIPS 2023 11

Making good decisions in partially observable and non-deterministic scenarios is a crucial capability for robots. A Partially Observable Markov Decision Process (POMDP) is a general framework for the above problem. Despi…

Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes

2013-02-06 · Anthony R. Cassandra, Michael L. Littman, Nevin Lianwen Zhang

Most exact algorithms for general partially observable Markov decision processes (POMDPs) use a form of dynamic programming in which a piecewise-linear and convex representation of one value function is transformed into …

HSVI can solve zero-sum Partially Observable Stochastic Games

2022-10-26 · Aurélien Delage, Olivier Buffet, Jilles S. Dibangoye, Abdallah Saffidine

State-of-the-art methods for solving 2-player zero-sum imperfect information games rely on linear programming or regret minimization, though not on dynamic programming (DP) or heuristic search (HS), while the latter are …

Decision Making Heuristic Search Open-Ended Question Answering Sequential Decision Making

Reinforcement Learning based on MPC/MHE for Unmodeled and Partially Observable Dynamics

2021-03-22 · Hossein Nejatbakhsh Esfahani, Arash Bahari Kordabad, Sebastien Gros

This paper proposes an observer-based framework for solving Partially Observable Markov Decision Processes (POMDPs) when an accurate model is not available. We first propose to use a Moving Horizon Estimation-Model Predi…

Model Predictive Control reinforcement-learning Reinforcement Learning (RL)

Variational Recurrent Models for Solving Partially Observable Control Tasks

2019-12-23 · ICLR 2020 1 · Dongqi Han, Kenji Doya, Jun Tani

In partially observable (PO) environments, deep reinforcement learning (RL) agents often suffer from unsatisfactory performance, since two problems need to be tackled together: how to extract information from the raw obs…

Deep Reinforcement Learning Memorization Reinforcement Learning Reinforcement Learning (RL)