$H_\infty$ Model-free Reinforcement Learning with Robust Stability Guarantee
Reinforcement learning is showing great potentials in robotics applications, including autonomous driving, robot manipulation and locomotion. However, with complex uncertainties in the real-world environment, it is difficult to guarantee the successful generalization and sim-to-real transfer of learned policies theoretically. In this paper, we introduce and extend the idea of robust stability and $H_\infty$ control to design policies with both stability and robustness guarantee. Specifically, a sample-based approach for analyzing the Lyapunov stability and performance robustness of a learning-based control system is proposed. Based on the theoretical results, a maximum entropy algorithm is developed for searching Lyapunov function and designing a policy with provable robust stability guarantee. Without any specific domain knowledge, our method can find a policy that is robust to various uncertainties and generalizes well to different test environments. In our experiments, we show that our method achieves better robustness to both large impulsive disturbances and parametric variations in the environment than the state-of-art results in both robust and generic RL, as well as classic control. Anonymous code is available to reproduce the experimental results at https://github.com/RobustStabilityGuaranteeRL/RobustStabilityGuaranteeRL.
Code (1)
Tasks
Autonomous Drivingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Robot ManipulationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Policy Optimization for $\mathcal{H}_{2}$ Linear Control with $\mathcal{H}_{\infty}$ Robustness Guarantee: Implicit Regularization and Global Convergence
Policy optimization (PO) is a key ingredient for modern reinforcement learning (RL). For control design, certain constraints are usually enforced on the policies to optimize, accounting for either the stability, robustne…
Reinforcement Learning (RL)Robust Policy Optimization in Continuous-time Mixed $\mathcal{H}_2/\mathcal{H}_\infty$ Stochastic Control
Following the recent resurgence in establishing linear control theoretic benchmarks for reinforcement leaning (RL)-based policy optimization (PO) for complex dynamical systems with continuous state and action spaces, an …
reinforcement-learningReinforcement Learning (RL)Multi-Objective $H_{\infty}$ Control for String Stability of Cooperative Adaptive Cruise Control Systems
Autonomous vehicle following systems are playing a decisive role to increase vehicle density on roads by shortening inter-vehicle time gaps. However, disturbance attenuation along a platoon of vehicles, i.e., string stab…
Infinity-norm-based Input-to-State-Stable Long Short-Term Memory networks: a thermal systems perspective
Recurrent Neural Networks (RNNs) have shown remarkable performances in system identification, particularly in nonlinear dynamical systems such as thermal processes. However, stability remains a critical challenge in prac…
Stability and $\mathcal{H}_{\infty}$ Performance Analysis of Stochastic Linear Networked and Quantized Control Systems
This paper studies the stability and $\mathcal{H}_{\infty}$ performance analysis problem for linear networked and quantized control systems with both communication delays random packet losses. To deal with the network-in…