Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions

2024-05-25 · Harry Zhang

Model-based Reinforcement Learning (MBRL) has shown many desirable properties for intelligent control tasks. However, satisfying safety and stability constraints during training and rollout remains an open question. We propose a new Model-based RL framework to enable efficient policy learning with unknown dynamics based on learning model predictive control (LMPC) framework with mathematically provable guarantees of stability. We introduce and explore a novel method for adding safety constraints for model-based RL during training and policy learning. The new stability-augmented framework consists of a neural-network-based learner that learns to construct a Lyapunov function, and a model-based RL agent to consistently complete the tasks while satisfying user-specified constraints given only sub-optimal demonstrations and sparse-cost feedback. We demonstrate the capability of the proposed framework through simulated experiments.

📄 PDF Abstract BibTeX arXiv:2405.16184

Code (1)

harryzhangOG/salved 공식 구현 tf

Tasks

Model-based Reinforcement LearningModel Predictive Controlreinforcement-learning Reinforcement Learning

Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions

2021-04-16 · Max H. Cohen, Calin Belta

This paper develops a model-based reinforcement learning (MBRL) framework for learning online the value function of an infinite-horizon optimal control problem while obeying safety constraints expressed as control barrie…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Safe Exploration

Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions

2023-05-16 · Desong Du, Shaohang Han, Naiming Qi, Haitham Bou Ammar 외

Reinforcement learning (RL) exhibits impressive performance when managing complicated control tasks for robots. However, its wide application to physical robots is limited by the absence of strong safety guarantees. To o…

reinforcement-learning Reinforcement Learning (RL)

Lyapunov-Guided Self-Alignment: Test-Time Adaptation for Offline Safe Reinforcement Learning

2026-04-29 · Seungyub Han, Hyungjin Kim, Jungwoo Lee arxiv

Offline reinforcement learning (RL) agents often fail when deployed, as the gap between training datasets and real environments leads to unsafe behavior. To address this, we present SAS (Self-Alignment for Safety), a tra…

Reinforcement Learning Test-time Adaptation Bayesian Inference

Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions

2020-04-16 · Jason Choi, Fernando Castañeda, Claire J. Tomlin, Koushil Sreenath

In this paper, the issue of model uncertainty in safety-critical control is addressed with a data-driven approach. For this purpose, we utilize the structure of an input-ouput linearization controller based on a nominal …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Achieving Safe Control Online through Integration of Harmonic Control Lyapunov-Barrier Functions with Unsafe Object-Centric Action Policies

2025-11-18 · Marlow Fawn, Matthias Scheutz arxiv

We propose a method for combining Harmonic Control Lyapunov-Barrier Functions (HCLBFs) derived from Signal Temporal Logic (STL) specifications with any given robot policy to turn an unsafe policy into a safe one with for…

Reinforcement Learning