Safe Reinforcement Learning using Ideas from Model Predictive Control
Reinforcement learning (RL) enables the synthesis of control policies directly from data, making it highly appealing for complex cyber-physical systems (CPSs) and robotics. A persistent challenge, however, is ensuring strict, hard safety constraints during the active learning phase. In real-world physical systems, violating mechanical limits can cause irreversible damage, necessitating that exploration remains strictly within safe operational regions. We propose a generalized framework that combines the adaptive, high-performance nature of deep reinforcement learning (DRL) with the formal safety guarantees of model predictive control (MPC). Using a mathematical model of the system dynamics, offline MPC computations define a feasible state-action space, representing all safe combinations of system states and control inputs that guarantee constraint satisfaction. During training and deployment, the RL agent's instantaneous actions are projected onto this globally verified feasible set via a safety filter. We systematically evaluate our generalized approach on a non-linear 1-DoF laboratory testbed, demonstrating successful exploration and stable policy convergence on physical hardware.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningActive LearningSimilar Papers 제목 키워드 기반
Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics
This paper proposes a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic…
Learning Theoryreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1DeepSafeMPC: Deep Learning-Based Model Predictive Control for Safe Multi-Agent Reinforcement Learning
Safe Multi-agent reinforcement learning (safe MARL) has increasingly gained attention in recent years, emphasizing the need for agents to not only optimize the global return but also adhere to safety requirements through…
Model Predictive ControlMuJoCoMulti-agent Reinforcement Learningreinforcement-learning+1Reinforcement Learning with Ensemble Model Predictive Safety Certification
Reinforcement learning algorithms need exploration to learn. However, unsupervised exploration prevents the deployment of such algorithms on safety-critical tasks and limits real-world deployment. In this paper, we propo…
Deep Reinforcement LearningmodelModel Predictive Controlreinforcement-learning+1Real-world validation of safe reinforcement learning, model predictive control and decision tree-based home energy management systems
Recent advancements in machine learning based energy management approaches, specifically reinforcement learning with a safety layer (OptLayerPolicy) and a metaheuristic algorithm generating a decision tree control policy…
energy managementManagementModel Predictive Controlreinforcement-learning+2Safe Reinforcement Learning with Chance-constrained Model Predictive Control
Real-world reinforcement learning (RL) problems often demand that agents behave safely by obeying a set of designed constraints. We address the challenge of safe RL by coupling a safety guide based on model predictive co…
Model Predictive Controlreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1