Virtuously Safe Reinforcement Learning

2018-05-29 · Henrik Aslund, El Mahdi El Mhamdi, Rachid Guerraoui, Alexandre Maurer

We show that when a third party, the adversary, steps into the two-party setting (agent and operator) of safely interruptible reinforcement learning, a trade-off has to be made between the probability of following the optimal policy in the limit, and the probability of escaping a dangerous situation created by the adversary. So far, the work on safely interruptible agents has assumed a perfect perception of the agent about its environment (no adversary), and therefore implicitly set the second probability to zero, by explicitly seeking a value of one for the first probability. We show that (1) agents can be made both interruptible and adversary-resilient, and (2) the interruptibility can be made safe in the sense that the agent itself will not seek to avoid it. We also solve the problem that arises when the agent does not go completely greedy, i.e. issues with safe exploration in the limit. Resilience to perturbed perception, safe exploration in the limit, and safe interruptibility are the three pillars of what we call \emph{virtuously safe reinforcement learning}.

📄 PDF Abstract BibTeX arXiv:1805.11447

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe ExplorationSafe Reinforcement Learning

Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Art

2026-01-08 · Timofey Tomashevskiy arxiv

This work provides a state-of-the-art survey of continual safe online reinforcement learning (COSRL) methods. We discuss theoretical aspects, challenges, and open questions in building continual online safe reinforcement…

Reinforcement Learning Continual Learning

Counterexample-Guided Repair of Reinforcement Learning Systems Using Safety Critics

2024-05-24 · David Boetius, Stefan Leue

Naively trained Deep Reinforcement Learning agents may fail to satisfy vital safety constraints. To avoid costly retraining, we may desire to repair a previously trained reinforcement learning agent to obviate unsafe beh…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning

Safe-RULE: Safe Reinforcement UnLEarning

2026-06-08 · Shixiong Jiang, Taozheng Zhu, Fanxin Kong arxiv

Offline safe reinforcement learning (Safe RL) enables policy learning without online interactions, making it suitable for safety-critical systems such as robotics systems. However, its reliance on static datasets exposes…

Reinforcement Learning

Provably Safe Reinforcement Learning from Analytic Gradients

2025-06-02 · Tim Walter, Hannah Markgraf, Jonathan Külz, Matthias Althoff

Deploying autonomous robots in safety-critical applications requires safety guarantees. Provably safe reinforcement learning is an active field of research which aims to provide such guarantees using safeguards. These sa…

reinforcement-learning Reinforcement Learning Safe Reinforcement Learning

SafeDojo: Safe Reinforcement Learning for VLA via Interactive World Model

2026-06-15 · Kai Tang, Peidong Jia, Zhong Chu, Jixian Wu 외 arxiv

Safe control is a prerequisite for real-world embodied intelligence, for which safe reinforcement learning has emerged as a promising paradigm. However, existing safe reinforcement learning methods either require costly …