Enter the Matrix: Safely Interruptible Autonomous Systems via Virtualization

2017-03-30 · Mark O. Riedl, Brent Harrison

Autonomous systems that operate around humans will likely always rely on kill switches that stop their execution and allow them to be remote-controlled for the safety of humans or to prevent damage to the system. It is theoretically possible for an autonomous system with sufficient sensor and effector capability that learn online using reinforcement learning to discover that the kill switch deprives it of long-term reward and thus learn to disable the switch or otherwise prevent a human operator from using the switch. This is referred to as the big red button problem. We present a technique that prevents a reinforcement learning agent from learning to disable the kill switch. We introduce an interruption process in which the agent's sensors and effectors are redirected to a virtual simulation where it continues to believe it is receiving reward. We illustrate our technique in a simple grid world environment.

📄 PDF Abstract BibTeX arXiv:1703.10284

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Virtuously Safe Reinforcement Learning

2018-05-29 · Henrik Aslund, El Mahdi El Mhamdi, Rachid Guerraoui, Alexandre Maurer

We show that when a third party, the adversary, steps into the two-party setting (agent and operator) of safely interruptible reinforcement learning, a trade-off has to be made between the probability of following the op…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Exploration+1

Pandora: A Cyber Range Environment for the Safe Testing and Deployment of Autonomous Cyber Attack Tools

2020-09-24 · Hetong Jiang, Taejun Choi, Ryan K. L. Ko

Cybersecurity tools are increasingly automated with artificial intelligent (AI) capabilities to match the exponential scale of attacks, compensate for the relatively slower rate of training new cybersecurity talents, and…

Efficiently Combining Human Demonstrations and Interventions for Safe Training of Autonomous Systems in Real-Time

2018-10-26 · Vinicius G. Goecks, Gregory M. Gremillion, Vernon J. Lawhern, John Valasek 외

This paper investigates how to utilize different forms of human interaction to safely train autonomous systems in real-time by learning from both human demonstrations and interventions. We implement two components of the…

Imitation Learning

Vision-based Navigation of Autonomous Vehicle in Roadway Environments with Unexpected Hazards

2018-09-27 · Mhafuzul Islam, Mahsrur Chowdhury, Hongda Li, Hongxin Hu

Vision-based navigation of autonomous vehicles primarily depends on the Deep Neural Network (DNN) based systems in which the controller obtains input from sensors/detectors, such as cameras and produces a vehicle control…

Autonomous Driving Autonomous Vehicles Navigate object-detection+3

Interruptible Algorithms for Multiproblem Solving

2018-10-26 · Spyros Angelopoulos, Alejandro Lopez-Ortiz

In this paper we address the problem of designing an interruptible system in a setting in which $n$ problem instances, all equally important, must be solved concurrently. The system involves scheduling executions of cont…

Scheduling