Black-Box Safety Validation of Autonomous Systems: A Multi-Fidelity Reinforcement Learning Approach
The increasing use of autonomous and semi-autonomous agents in society has made it crucial to validate their safety. However, the complex scenarios in which they are used may make formal verification impossible. To address this challenge, simulation-based safety validation is employed to test the complex system. Recent approaches using reinforcement learning are prone to excessive exploitation of known failures and a lack of coverage in the space of failures. To address this limitation, a type of Markov decision process called the "knowledge MDP" has been defined. This approach takes into account both the learned model and its metadata, such as sample counts, in estimating the system's knowledge through the "knows what it knows" framework. A novel algorithm that extends bidirectional learning to multiple fidelities of simulators has been developed to solve the safety validation problem. The effectiveness of this approach is demonstrated through a case study in which an adversary is trained to intercept a test model in a grid-world environment. Monte Carlo trials compare the sample efficiency of the proposed algorithm to learning with a single-fidelity simulator and show the importance of incorporating knowledge about learned models into the decision-making process.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingReinforcement Learning (RL)Similar Papers 제목 키워드 기반
A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems
Autonomous cyber-physical systems (CPS) can improve safety and efficiency for safety-critical applications, but require rigorous testing before deployment. The complexity of these systems often precludes the use of forma…
Autonomous VehiclesCollision AvoidanceProblem DecompositionSurveyBayesian Safety Validation for Failure Probability Estimation of Black-Box Systems
Estimating the probability of failure is an important step in the certification of safety-critical systems. Efficient estimation methods are often needed due to the challenges posed by high-dimensional input spaces, risk…
Bayesian OptimizationDecision MakingDiffusion-Based Failure Sampling for Evaluating Safety-Critical Autonomous Systems
Validating safety-critical autonomous systems in high-dimensional domains such as robotics presents a significant challenge. Existing black-box approaches based on Markov chain Monte Carlo may require an enormous number …
DenoisingTask PlanningTransfer Learning for Efficient Iterative Safety Validation
Safety validation is important during the development of safety-critical autonomous systems but can require significant computational effort. Existing algorithms often start from scratch each time the system under test c…
Autonomous Drivingreinforcement-learningReinforcement Learning (RL)Transfer LearningDiffusion Models for Safety Validation of Autonomous Driving Systems
Safety validation of autonomous driving systems is extremely challenging due to the high risks and costs of real-world testing as well as the rarity and diversity of potential failures. To address these challenges, we tr…
Autonomous DrivingDenoisingDiversity