Predictive Safety Shield for Dyna-Q Reinforcement Learning

2025-11-26 · Jin Pin, Krasowski Hanna, Vanneaux Elena arxiv

Obtaining safety guarantees for reinforcement learning is a major challenge to achieve applicability for real-world tasks. Safety shields extend standard reinforcement learning and achieve hard safety guarantees. However, existing safety shields commonly use random sampling of safe actions or a fixed fallback controller, therefore disregarding future performance implications of different safe actions. In this work, we propose a predictive safety shield for model-based reinforcement learning agents in discrete space. Our safety shield updates the Q-function locally based on safe predictions, which originate from a safe simulation of the environment model. This shielding approach improves performance while maintaining hard safety guarantees. Our experiments on gridworld environments demonstrate that even short prediction horizons can be sufficient to identify the optimal path. We observe that our approach is robust to distribution shifts, e.g., between simulation and reality, without requiring additional training.

📄 PDF Abstract BibTeX arXiv:2511.21531

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Safe Reinforcement Learning with Nonlinear Dynamics via Model Predictive Shielding

2019-05-25 · Osbert Bastani

Reinforcement learning is a promising approach to synthesizing policies for challenging robotics tasks. A key problem is how to ensure safety of the learned policy---e.g., that a walking robot does not fall over or that …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Reinforcement Learning

Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learning

2024-05-22 · Arko Banerjee, Kia Rahmani, Joydeep Biswas, Isil Dillig

Among approaches for provably safe reinforcement learning, Model Predictive Shielding (MPS) has proven effective at complex tasks in continuous, high-dimensional state spaces, by leveraging a backup policy to ensure safe…

reinforcement-learning Reinforcement Learning Safe Reinforcement Learning

Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics

2019-10-24 · Shuo Li, Osbert Bastani

This paper proposes a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic…

Learning Theory reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

MAMPS: Safe Multi-Agent Reinforcement Learning via Model Predictive Shielding

2019-10-25 · Wenbo Zhang, Osbert Bastani, Vijay Kumar

Reinforcement learning is a promising approach to learning control policies for performing complex multi-agent robotics tasks. However, a policy learned in simulation often fails to guarantee even simple safety propertie…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Model-based Dynamic Shielding for Safe and Efficient Multi-Agent Reinforcement Learning

2023-04-13 · Wenli Xiao, Yiwei Lyu, John Dolan

Multi-Agent Reinforcement Learning (MARL) discovers policies that maximize reward but do not have safety guarantees during the learning and deployment phases. Although shielding with Linear Temporal Logic (LTL) is a prom…