Stabilizing reinforcement learning control: A modular framework for optimizing over all stable behavior

2023-10-21 · Nathan P. Lawrence, Philip D. Loewen, Shuyuan Wang, Michael G. Forbes, R. Bhushan Gopaluni

We propose a framework for the design of feedback controllers that combines the optimization-driven and model-free advantages of deep reinforcement learning with the stability guarantees provided by using the Youla-Kucera parameterization to define the search domain. Recent advances in behavioral systems allow us to construct a data-driven internal model; this enables an alternative realization of the Youla-Kucera parameterization based entirely on input-output exploration data. Perhaps of independent interest, we formulate and analyze the stability of such data-driven models in the presence of noise. The Youla-Kucera approach requires a stable "parameter" for controller design. For the training of reinforcement learning agents, the set of all stable linear operators is given explicitly through a matrix factorization approach. Moreover, a nonlinear extension is given using a neural network to express a parameterized set of stable operators, which enables seamless integration with standard deep learning libraries. Finally, we show how these ideas can also be applied to tune fixed-structure controllers.

📄 PDF Abstract BibTeX arXiv:2310.14098

Code (1)

nplawrence/stablebehavior.jl 공식 구현

Tasks

AllDeep Reinforcement Learningreinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

A modular framework for stabilizing deep reinforcement learning control

2023-04-07 · Nathan P. Lawrence, Philip D. Loewen, Shuyuan Wang, Michael G. Forbes 외

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning

Data-Based Efficient Off-Policy Stabilizing Optimal Control Algorithms for Discrete-Time Linear Systems via Damping Coefficients

2024-12-30 · Dongdong Li, Jiuxiang Dong

Policy iteration is one of the classical frameworks of reinforcement learning, which requires a known initial stabilizing control. However, finding the initial stabilizing control depends on the known system model. To re…

Q-Learning reinforcement-learning Reinforcement Learning

Modular Control Architecture for Safe Marine Navigation: Reinforcement Learning and Predictive Safety Filters

2023-12-04 · Aksel Vaaler, Svein Jostein Husa, Daniel Menges, Thomas Nakken Larsen 외

Many autonomous systems face safety challenges, requiring robust closed-loop control to handle physical limitations and safety constraints. Real-world systems, like autonomous ships, encounter nonlinear dynamics and envi…

A note on stabilizing reinforcement learning

2021-11-24 · Pavel Osinenko, Grigory Yaremenko, Ilya Osokin

Reinforcement learning is a general methodology of adaptive optimal control that has attracted much attention in various fields ranging from video game industry to robot manipulators. Despite its remarkable performance d…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems

2022-06-03 · Sahin Lale, Yuanyuan Shi, Guannan Qu, Kamyar Azizzadenesheli 외

Learning a dynamical system requires stabilizing the unknown dynamics to avoid state blow-ups. However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the …

reinforcement-learning Reinforcement Learning (RL)