Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning

2017-04-10 · NeurIPS 2017 12 · El Mahdi El Mhamdi, Rachid Guerraoui, Hadrien Hendrikx, Alexandre Maurer

In reinforcement learning, agents learn by performing actions and observing their outcomes. Sometimes, it is desirable for a human operator to \textit{interrupt} an agent in order to prevent dangerous situations from happening. Yet, as part of their learning process, agents may link these interruptions, that impact their reward, to specific states and deliberately avoid them. The situation is particularly challenging in a multi-agent context because agents might not only learn from their own past interruptions, but also from those of other agents. Orseau and Armstrong defined \emph{safe interruptibility} for one learner, but their work does not naturally extend to multi-agent systems. This paper introduces \textit{dynamic safe interruptibility}, an alternative definition more suited to decentralized learning problems, and studies this notion in two learning frameworks: \textit{joint action learners} and \textit{independent learners}. We give realistic sufficient conditions on the learning algorithm to enable dynamic safe interruptibility in the case of joint action learners, yet show that these conditions are not sufficient for independent learners. We show however that if agents can detect interruptions, it is possible to prune the observations to ensure dynamic safe interruptibility even for independent learners.

📄 PDF Abstract BibTeX arXiv:1704.02882

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Virtuously Safe Reinforcement Learning

2018-05-29 · Henrik Aslund, El Mahdi El Mhamdi, Rachid Guerraoui, Alexandre Maurer

We show that when a third party, the adversary, steps into the two-party setting (agent and operator) of safely interruptible reinforcement learning, a trade-off has to be made between the probability of following the op…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Exploration+1

AI Safety Gridworlds

2017-11-27 · Jan Leike, Miljan Martic, Victoria Krakovna, Pedro A. Ortega 외

We present a suite of reinforcement learning environments illustrating various safety properties of intelligent agents. These problems include safe interruptibility, avoiding side effects, absent supervisor, reward gamin…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Decentralized Contingency MPC based on Safe Sets for Nonlinear Multi-agent Collision Avoidance

2026-05-11 · Max Studt, Georg Schildbach arxiv

Decentralized collision avoidance remains challenging, particularly when agents do not communicate any information related to planned trajectories. Most existing approaches either rely on conservative coordination mechan…

Collision Avoidance

Risk-aware Safe Control for Decentralized Multi-agent Systems via Dynamic Responsibility Allocation

2023-05-22 · Yiwei Lyu, Wenhao Luo, John M. Dolan

Decentralized control schemes are increasingly favored in various domains that involve multi-agent systems due to the need for computational efficiency as well as general applicability to large-scale systems. However, in…

Autonomous Driving Computational Efficiency

MAD-PINN: A Decentralized Physics-Informed Machine Learning Framework for Safe and Optimal Multi-Agent Control

2025-09-28 · Manan Tayal, Aditya Singh, Shishir Kolathaya, Somil Bansal arxiv

Co-optimizing safety and performance in large-scale multi-agent systems remains a fundamental challenge. Existing approaches based on multi-agent reinforcement learning (MARL), safety filtering, or Model Predictive Contr…

Multi-agent Reinforcement Learning