Safe Reinforcement Learning using Data-Driven Predictive Control

2022-11-20 · Mahmoud Selim, Amr Alanwar, M. Watheq El-Kharashi, Hazem M. Abbas, Karl H. Johansson

Reinforcement learning (RL) algorithms can achieve state-of-the-art performance in decision-making and continuous control tasks. However, applying RL algorithms on safety-critical systems still needs to be well justified due to the exploration nature of many RL algorithms, especially when the model of the robot and the environment are unknown. To address this challenge, we propose a data-driven safety layer that acts as a filter for unsafe actions. The safety layer uses a data-driven predictive controller to enforce safety guarantees for RL policies during training and after deployment. The RL agent proposes an action that is verified by computing the data-driven reachability analysis. If there is an intersection between the reachable set of the robot using the proposed action, we call the data-driven predictive controller to find the closest safe action to the proposed unsafe action. The safety layer penalizes the RL agent if the proposed action is unsafe and replaces it with the closest safe one. In the simulation, we show that our method outperforms state-of-the-art safe RL methods on the robotics navigation problem for a Turtlebot 3 in Gazebo and a quadrotor in Unreal Engine 4 (UE4).

📄 PDF Abstract BibTeX arXiv:2211.11027

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Reinforcement Learning

A predictive safety filter for learning-based control of constrained nonlinear dynamical systems

2018-12-13 · Kim P. Wabersich, Melanie N. Zeilinger

The transfer of reinforcement learning (RL) techniques into real-world applications is challenged by safety requirements in the presence of physical limitations. Most RL methods, in particular the most popular algorithms…

Model Predictive Control Reinforcement Learning Reinforcement Learning (RL)Safe Exploration

DeepSafeMPC: Deep Learning-Based Model Predictive Control for Safe Multi-Agent Reinforcement Learning

2024-03-11 · Xuefeng Wang, Henglin Pu, Hyung Jun Kim, Husheng Li

Safe Multi-agent reinforcement learning (safe MARL) has increasingly gained attention in recent years, emphasizing the need for agents to not only optimize the global return but also adhere to safety requirements through…

Model Predictive Control MuJoCo Multi-agent Reinforcement Learning reinforcement-learning+1

Safe Chance Constrained Reinforcement Learning for Batch Process Control

2021-04-23 · Max Mowbray, Panagiotis Petsagkourakis, Ehecatl Antonio del Río Chanona, Dongda Zhang

Reinforcement Learning (RL) controllers have generated excitement within the control community. The primary advantage of RL controllers relative to existing methods is their ability to optimize uncertain systems independ…

Gaussian Processes Model Predictive Control reinforcement-learning Reinforcement Learning+1

Reinforcement Learning with Ensemble Model Predictive Safety Certification

2024-02-06 · Sven Gronauer, Tom Haider, Felippe Schmoeller da Roza, Klaus Diepold

Reinforcement learning algorithms need exploration to learn. However, unsupervised exploration prevents the deployment of such algorithms on safety-critical tasks and limits real-world deployment. In this paper, we propo…

Deep Reinforcement Learning model Model Predictive Control reinforcement-learning+1

Knowledge-Data-Dual-Driven Reinforcement Learning for Autonomous Vehicle Control in Mixed Traffic

2026-08-14 · Jie Fang, Wei Zheng, Mengyun Xu, Eui-Jin Kim arxiv

In mixed traffic, decision-making for autonomous vehicles (AVs) confronts three interrelated challenges. First, physics-based priors incorporated into reinforcement learning (RL) models fail to capture latent interactive…

Reinforcement Learning Autonomous Vehicles