Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems
Neural network controllers have become popular in control tasks thanks to their flexibility and expressivity. Stability is a crucial property for safety-critical dynamical systems, while stabilization of partially observed systems, in many cases, requires controllers to retain and process long-term memories of the past. We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems, and derive convex stability conditions based on integral quadratic constraints, S-lemma and sequential convexification. To ensure stability during the learning and control process, we propose a projected policy gradient method that iteratively enforces the stability conditions in the reparametrized space taking advantage of mild additional information on system dynamics. Numerical experiments show that our method learns stabilizing controllers while using fewer samples and achieving higher final performance compared with policy gradient.
Code (1)
Tasks
LEMMASimilar Papers 제목 키워드 기반
Synthesis of Stabilizing Recurrent Equilibrium Network Controllers
We propose a parameterization of a nonlinear dynamic controller based on the recurrent equilibrium network, a generalization of the recurrent neural network. We derive constraints on the parameterization under which the …
Policy Gradient MethodsLearning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear System
This paper proposes a nonlinear policy architecture for control of partially-observed linear dynamical systems providing built-in closed-loop stability guarantees. The policy is based on a nonlinear version of the Youla …
AllReinforcement Learning (RL)Robust Optimal Safe and Stability Guaranteeing Reinforcement Learning Control for Quadcopter
Recent advances in deep learning have provided new data-driven ways of controller design to replace the traditional manual synthesis and certification approaches. Employing neural network (NN) as controllers however, pre…
Online Learning for Supervisory Switching Control
We study supervisory switching control for partially-observed linear dynamical systems. The objective is to identify and deploy a suitable controller for the unknown system by periodically selecting among a collection of…
A Nonlinear Separation Principle via Contraction Theory: Applications to Neural Networks, Control, and Learning
This paper establishes a nonlinear separation principle based on contraction theory and derives sharp stability conditions for recurrent neural networks (RNNs). First, we introduce a nonlinear separation principle that g…
Image Classification