Tolerance of Reinforcement Learning Controllers against Deviations in Cyber Physical Systems
Cyber-physical systems (CPS) with reinforcement learning (RL)-based controllers are increasingly being deployed in complex physical environments such as autonomous vehicles, the Internet-of-Things(IoT), and smart cities. An important property of a CPS is tolerance; i.e., its ability to function safely under possible disturbances and uncertainties in the actual operation. In this paper, we introduce a new, expressive notion of tolerance that describes how well a controller is capable of satisfying a desired system requirement, specified using Signal Temporal Logic (STL), under possible deviations in the system. Based on this definition, we propose a novel analysis problem, called the tolerance falsification problem, which involves finding small deviations that result in a violation of the given requirement. We present a novel, two-layer simulation-based analysis framework and a novel search heuristic for finding small tolerance violations. To evaluate our approach, we construct a set of benchmark problems where system parameters can be configured to represent different types of uncertainties and disturbancesin the system. Our evaluation shows that our falsification approach and heuristic can effectively find small tolerance violations.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous VehiclesReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Investigating Robustness in Cyber-Physical Systems: Specification-Centric Analysis in the face of System Deviations
The adoption of cyber-physical systems (CPS) is on the rise in complex physical environments, encompassing domains such as autonomous vehicles, the Internet of Things (IoT), and smart cities. A critical attribute of CPS …
AttributeAutonomous VehiclesReinforcement Learning (RL)Multi-Agent Deep Reinforcement Learning-Driven Mitigation of Adverse Effects of Cyber-Attacks on Electric Vehicle Charging Station
An electric vehicle charging station (EVCS) infrastructure is the backbone of transportation electrification. However, the EVCS has myriads of exploitable vulnerabilities in software, hardware, supply chain, and incumben…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Model-Free Reinforcement Learning Control for Resilient Cyber-Physical Systems
This paper compares the performance of model-free controllers on a nonlinear system under cyberattacks, including false data injection and denial-of-service attacks. Four RL reward types are analyzed for accuracy, cost, …
Reinforcement LearningDiscovery of False Data Injection Schemes on Frequency Controllers with Reinforcement Learning
While inverter-based distributed energy resources (DERs) play a crucial role in integrating renewable energy into the power system, they concurrently diminish the grid's system inertia, elevating the risk of frequency in…
Reinforcement Learning (RL)On tolerance of discrete systems with respect to transition perturbations
Control systems should enforce a desired property for both expected modeled situations as well as unexpected unmodeled environmental situations. Existing methods focus on designing controllers to enforce the desired prop…