Reinforcement Learning for Control with Probabilistic Stability Guarantee

2021-01-01 · Minghao Han, Zhipeng Zhou, Lixian Zhang, Jun Wang, Wei Pan

Reinforcement learning is promising to control dynamical systems for which the traditional control methods are hardly applicable. However, in control theory, the stability of a closed-loop system can be hardly guaranteed using the policy/controller learned solely from samples. In this paper, we will combine Lyapunov's method in control theory and stochastic analysis to analyze the mean square stability of MDP in a model-free manner. Furthermore, the finite sample bounds on the probability of stability are derived as a function of the number M and length T of the sampled trajectories. And we show that there is a lower bound on T and the probability is much more demanding for M than T. Based on the theoretical results, a REINFORCE like algorithm is proposed to learn the controller and the Lyapunov function simultaneously.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

REINFORCE REINFORCE is a Monte Carlo variant of a policy gradient algorithm in reinforcement learning. The agent collects samples of an episode using its current policy, and uses it to…

Reinforcement Learning for Control with Probabilistic Stability Guarantee: A Finite-Sample Approach

2026-02-09 · Minghao Han, Lixian Zhang, Chenliang Liu, Zhipeng Zhou 외 arxiv

This paper presents a novel approach to reinforcement learning (RL) for control systems that provides probabilistic stability guarantees using finite data. Leveraging Lyapunov's method, we propose a probabilistic stabili…

Reinforcement Learning

Stabilizing Off-Policy Reinforcement Learning with Conservative Policy Gradients

2019-09-25 · Chen Tessler, Nadav Merlis, Shie Mannor

In recent years, advances in deep learning have enabled the application of reinforcement learning algorithms in complex domains. However, they lack the theoretical guarantees which are present in the tabular setting and …

Deep Reinforcement Learning MuJoCo reinforcement-learning Reinforcement Learning+1

Reinforcement Learning for Distributed Transient Frequency Control with Stability and Safety Guarantees

2022-07-07 · Zhenyi Yuan, Changhong Zhao, Jorge Cortes

This paper proposes a reinforcement learning-based approach for optimal transient frequency control in power systems with stability and safety guarantees. Building on Lyapunov stability theory and safety-critical control…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Probabilistically Input-to-State Stable Stochastic Model Predictive Control

2024-10-10 · Maik Pfefferkorn, Rolf Findeisen

Employing model predictive control to systems with unbounded, stochastic disturbances poses the challenge of guaranteeing safety, i.e., repeated feasibility and stability of the closed-loop system. Especially, there are …

Model Predictive Control

Probabilistic Guarantees for Safe Deep Reinforcement Learning

2020-05-14 · Edoardo Bacci, David Parker

Deep reinforcement learning has been successfully applied to many control tasks, but the application of such agents in safety-critical scenarios has been limited due to safety concerns. Rigorous testing of these controll…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)