Safe Model-based Reinforcement Learning with Stability Guarantees

2017-05-23 · NeurIPS 2017 12 · Felix Berkenkamp, Matteo Turchetta, Angela P. Schoellig, Andreas Krause

Reinforcement learning is a powerful paradigm for learning optimal policies from experimental data. However, to find optimal policies, most reinforcement learning algorithms explore all possible actions, which may be harmful for real-world systems. As a consequence, learning algorithms are rarely applied on safety-critical systems in the real world. In this paper, we present a learning algorithm that explicitly considers safety, defined in terms of stability guarantees. Specifically, we extend control-theoretic results on Lyapunov stability verification and show how to use statistical models of the dynamics to obtain high-performance control policies with provable stability certificates. Moreover, under additional regularity assumptions in terms of a Gaussian process prior, we prove that one can effectively and safely collect data in order to learn about the dynamics and thus both improve control performance and expand the safe region of the state space. In our experiments, we show how the resulting algorithm can safely optimize a neural network policy on a simulated inverted pendulum, without the pendulum ever falling down.

📄 PDF Abstract BibTeX arXiv:1705.08551

Code (1)

befelix/safe_learning 공식 구현 tf

Tasks

modelModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reinforcement Learning for Distributed Transient Frequency Control with Stability and Safety Guarantees

2022-07-07 · Zhenyi Yuan, Changhong Zhao, Jorge Cortes

This paper proposes a reinforcement learning-based approach for optimal transient frequency control in power systems with stability and safety guarantees. Building on Lyapunov stability theory and safety-critical control…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Physical Deep Reinforcement Learning Towards Safety Guarantee

2023-03-29 · Hongpeng Cao, Yanbing Mao, Lui Sha, Marco Caccamo

Deep reinforcement learning (DRL) has achieved tremendous success in many complex decision-making tasks of autonomous systems with high-dimensional state and/or action spaces. However, the safety and stability still rema…

Decision Making Deep Reinforcement Learning reinforcement-learning Reinforcement Learning

Robust Optimal Safe and Stability Guaranteeing Reinforcement Learning Control for Quadcopter

2024-12-18 · Sanghyoup Gu, Ratnesh Kumar

Recent advances in deep learning have provided new data-driven ways of controller design to replace the traditional manual synthesis and certification approaches. Employing neural network (NN) as controllers however, pre…

Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions

2024-05-25 · Harry Zhang

Model-based Reinforcement Learning (MBRL) has shown many desirable properties for intelligent control tasks. However, satisfying safety and stability constraints during training and rollout remains an open question. We p…

Model-based Reinforcement Learning Model Predictive Control reinforcement-learning Reinforcement Learning

Lyapunov-Aware Quantum-Inspired Reinforcement Learning for Continuous-Time Vehicle Control: A Feasibility Study

2025-10-21 · Nutkritta Kraipatthanapong, Natthaphat Thathong, Pannita Suksawas, Thanunnut Klunklin 외 arxiv

This paper presents a novel Lyapunov-Based Quantum Reinforcement Learning (LQRL) framework that integrates quantum policy optimization with Lyapunov stability analysis for continuous-time vehicle control. The proposed ap…

Reinforcement Learning