Runtime Safety Assurance Using Reinforcement Learning

2020-10-20 · Christopher Lazarus, James G. Lopez, Mykel J. Kochenderfer

The airworthiness and safety of a non-pedigreed autopilot must be verified, but the cost to formally do so can be prohibitive. We can bypass formal verification of non-pedigreed components by incorporating Runtime Safety Assurance (RTSA) as mechanism to ensure safety. RTSA consists of a meta-controller that observes the inputs and outputs of a non-pedigreed component and verifies formally specified behavior as the system operates. When the system is triggered, a verified recovery controller is deployed. Recovery controllers are designed to be safe but very likely disruptive to the operational objective of the system, and thus RTSA systems must balance safety and efficiency. The objective of this paper is to design a meta-controller capable of identifying unsafe situations with high accuracy. High dimensional and non-linear dynamics in which modern controllers are deployed along with the black-box nature of the nominal controllers make this a difficult problem. Current approaches rely heavily on domain expertise and human engineering. We frame the design of RTSA with the Markov decision process (MDP) framework and use reinforcement learning (RL) to solve it. Our learned meta-controller consistently exhibits superior performance in our experiments compared to our baseline, human engineered approach.

📄 PDF Abstract BibTeX arXiv:2010.10618

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Searching for Optimal Runtime Assurance via Reachability and Reinforcement Learning

2023-10-06 · Kristina Miller, Christopher K. Zeitler, William Shen, Kerianne Hobbs 외

A runtime assurance system (RTA) for a given plant enables the exercise of an untrusted or experimental controller while assuring safety with a backup (or safety) controller. The relevant computational design problem is …

reinforcement-learning Reinforcement Learning

A Subjective Logic-based method for runtime confidence updates in safety arguments

2026-05-21 · Benjamin Herd, Jessica Kelly, Clarissa Heinemann, João-Vitor Zacchi arxiv

We present a method for dynamic quantitative assurance that enhances static safety cases with continuous, runtime-driven confidence updates. The method quantifies and propagates confidence across the development lifecycl…

Towards Continuous Assurance with Formal Verification and Assurance Cases

2025-11-17 · Dhaminda B. Abeywickrama, Michael Fisher, Frederic Wheeler, Louise Dennis arxiv

Autonomous systems must sustain justified confidence in their correctness and safety across their operational lifecycle-from design and deployment through post-deployment evolution. Traditional assurance methods often se…

Stacked Universal Successor Feature Approximators for Safety in Reinforcement Learning

2024-09-06 · Ian Cannon, Washington Garcia, Thomas Gresavage, Joseph Saurine 외

Real-world problems often involve complex objective structures that resist distillation into reinforcement learning environments with a single objective. Operation costs must be balanced with multi-dimensional task perfo…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning

Synergistic Simplex: Cooperative Runtime Assurance for Safety-Critical Autonomous Systems

2026-05-05 · Ayoosh Bansal, Mikael Yeghiazaryan, Artyom Khachatryan, Tianyi Zhu 외 arxiv

Autonomous systems increasingly rely on machine-learning (ML) components for safety-critical tasks such as perception and control in autonomous vehicles (AVs). While ML enables essential capabilities, it inevitably exhib…

Autonomous Vehicles