Reduced-Dimensional Reinforcement Learning Control using Singular Perturbation Approximations
We present a set of model-free, reduced-dimensional reinforcement learning (RL) based optimal control designs for linear time-invariant singularly perturbed (SP) systems. We first present a state-feedback and output-feedback based RL control design for a generic SP system with unknown state and input matrices. We take advantage of the underlying time-scale separation property of the plant to learn a linear quadratic regulator (LQR) for only its slow dynamics, thereby saving a significant amount of learning time compared to the conventional full-dimensional RL controller. We analyze the sub-optimality of the design using SP approximation theorems and provide sufficient conditions for closed-loop stability. Thereafter, we extend both designs to clustered multi-agent consensus networks, where the SP property reflects through clustering. We develop both centralized and cluster-wise block-decentralized RL controllers for such networks, in reduced dimensions. We demonstrate the details of the implementation of these controllers using simulations of relevant numerical examples and compare them with conventional RL designs to show the computational benefits of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Clusteringreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Singular Perturbation-based Reinforcement Learning of Two-Point Boundary Optimal Control Systems
This work presents a technique for learning systems, where the learning process is guided by knowledge of the physics of the system. In particular, we solve the problem of the two-point boundary optimal control problem o…
reinforcement-learningReinforcement Learning (RL)Approximate Hamilton-Jacobi Reachability Analysis for a Class of Two-Timescale Systems, with Application to Biological Models
Hamilton-Jacobi reachability (HJR) is an exciting framework used for control of safety-critical systems with nonlinear and possibly uncertain dynamics. However, HJR suffers from the curse of dimensionality, with computat…
Infinitesimal Higher-Order Spectral Variations in Rectangular Real Random Matrices
We present a theoretical framework for deriving the general $n$-th order Fr\'echet derivatives of singular values in real rectangular matrices, by leveraging reduced resolvent operators from Kato's analytic perturbation …
Model Reduction of Converter-Dominated Power Systems by Singular Perturbation Theory
The increasing integration of power electronic devices is driving the development of more advanced tools and methods for the modeling, analysis, and control of modern power systems to cope with the different time-scale o…
Beamspace Multidimensional ESPRIT Approaches for Simultaneous Localization and Communications
Modern wireless communication systems operating at high carrier frequencies are characterized by a high dimensionality of the underlying parameter space (including channel gains, angles, delays, and possibly Doppler shif…