Statistical Verification of Autonomous Systems using Surrogate Models and Conformal Inference
In this paper, we propose conformal inference based approach for statistical verification of CPS models. Cyber-physical systems (CPS) such as autonomous vehicles, avionic systems, and medical devices operate in highly uncertain environments. This uncertainty is typically modeled using a finite number of parameters or input signals. Given a system specification in Signal Temporal Logic (STL), we would like to verify that for all (infinite) values of the model parameters/input signals, the system satisfies its specification. Unfortunately, this problem is undecidable in general. {\em Statistical model checking} (SMC) offers a solution by providing guarantees on the correctness of CPS models by statistically reasoning on model simulations. We propose a new approach for statistical verification of CPS models for user-provided distribution on the model parameters. Our technique uses model simulations to learn {\em surrogate models}, and uses {\em conformal inference} to provide probabilistic guarantees on the satisfaction of a given STL property. Additionally, we can provide prediction intervals containing the quantitative satisfaction values of the given STL property for any user-specified confidence level. We also propose a refinement procedure based on Gaussian Process (GP)-based surrogate models for obtaining fine-grained probabilistic guarantees over sub-regions in the parameter space. This in turn enables the CPS designer to choose assured validity domains in the parameter space for safety-critical applications. Finally, we demonstrate the efficacy of our technique on several CPS models.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous VehiclesPrediction IntervalsSimilar Papers 제목 키워드 기반
Formal Verification and Control with Conformal Prediction
In this survey, we design formal verification and control algorithms for autonomous systems with practical safety guarantees using conformal prediction (CP), a statistical tool for uncertainty quantification. We focus on…
ArticlesConformal PredictionPredictionRobot Navigation+2Deterministic World Models for Verification of Closed-loop Vision-based Systems
Verifying closed-loop vision-based control systems remains a fundamental challenge due to the high dimensionality of images and the difficulty of modeling visual environments. While generative models are increasingly use…
State-Dependent Conformal Perception Bounds for Neuro-Symbolic Verification of Autonomous Systems
It remains a challenge to provide safety guarantees for autonomous systems with neural perception and control. A typical approach obtains symbolic bounds on perception error (e.g., using conformal prediction) and perform…
Conformal PredictionGuaranteed confidence-band enclosures for PDE surrogates
We propose a method for obtaining statistically guaranteed confidence bands for functional machine learning techniques: surrogate models which map between function spaces, motivated by the need build reliable PDE emulato…
PredictionPhysics Constrained Motion Prediction with Uncertainty Quantification
Predicting the motion of dynamic agents is a critical task for guaranteeing the safety of autonomous systems. A particular challenge is that motion prediction algorithms should obey dynamics constraints and quantify pred…
Autonomous DrivingAutonomous RacingConformal Predictionmotion prediction+3