Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems

2024-04-23 · Haozhe Tian, Homayoun Hamedmoghadam, Robert Shorten, Pietro Ferraro

Reinforcement Learning (RL) is a powerful method for controlling dynamic systems, but its learning mechanism can lead to unpredictable actions that undermine the safety of critical systems. Here, we propose RL with Adaptive Regularization (RL-AR), an algorithm that enables safe RL exploration by combining the RL policy with a policy regularizer that hard-codes the safety constraints. RL-AR performs policy combination via a "focus module," which determines the appropriate combination depending on the state--relying more on the safe policy regularizer for less-exploited states while allowing unbiased convergence for well-exploited states. In a series of critical control applications, we demonstrate that RL-AR not only ensures safety during training but also achieves a return competitive with the standards of model-free RL that disregards safety.

📄 PDF Abstract BibTeX arXiv:2404.15199

Code (1)

haozhetian/rl-ar 공식 구현 pytorch

Tasks

Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Focus 설명 없음

Adaptive Aggregation for Safety-Critical Control

2023-02-07 · Huiliang Zhang, Di wu, Benoit Boulet

Safety has been recognized as the central obstacle to preventing the use of reinforcement learning (RL) for real-world applications. Different methods have been developed to deal with safety concerns in RL. However, lear…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Reinforcement Learning+1

Safe Reinforcement Learning Using Robust Action Governor

2021-02-21 · Yutong Li, Nan Li, H. Eric Tseng, Anouck Girard 외

Reinforcement Learning (RL) is essentially a trial-and-error learning procedure which may cause unsafe behavior during the exploration-and-exploitation process. This hinders the application of RL to real-world control pr…

RAG reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Self-Organizing Dual-Buffer Adaptive Clustering Experience Replay (SODACER) for Safe Reinforcement Learning in Optimal Control

2026-01-10 · Roya Khalili Amirabadi, Mohsen Jalaeian Farimani, Omid Solaymani Fard arxiv

This paper proposes a novel reinforcement learning framework, named Self-Organizing Dual-buffer Adaptive Clustering Experience Replay (SODACER), designed to achieve safe and scalable optimal control of nonlinear systems.…

Reinforcement Learning

SafeLight: A Reinforcement Learning Method toward Collision-free Traffic Signal Control

2022-11-20 · Wenlu Du, Junyi Ye, Jingyi Gu, Jing Li 외

Traffic signal control is safety-critical for our daily life. Roughly one-quarter of road accidents in the U.S. happen at intersections due to problematic signal timing, urging the development of safety-oriented intersec…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Reinforcement Learning+1

Safe Deep Policy Adaptation

2023-10-08 · Wenli Xiao, Tairan He, John Dolan, Guanya Shi

A critical goal of autonomy and artificial intelligence is enabling autonomous robots to rapidly adapt in dynamic and uncertain environments. Classic adaptive control and safe control provide stability and safety guarant…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Reinforcement Learning