Modulation-Enhanced Excitation for Continuous-Time Reinforcement Learning via Symmetric Kronecker Products
This work introduces new results in continuous-time reinforcement learning (CT-RL) control of affine nonlinear systems to address a major algorithmic challenge due to a lack of persistence of excitation (PE). This PE design limitation has previously stifled CT-RL numerical performance and prevented these algorithms from achieving control synthesis goals. Our new theoretical developments in symmetric Kronecker products enable a proposed modulation-enhanced excitation (MEE) framework to make PE significantly more systematic and intuitive to achieve for real-world designers. MEE is applied to the suite of recently-developed excitable integral reinforcement learning (EIRL) algorithms, yielding a class of enhanced high-performance CT-RL control design methods which, due to the symmetric Kronecker product algebra, retain EIRL's convergence and closed-loop stability guarantees. Through numerical evaluation studies, we demonstrate how our new MEE framework achieves substantial improvements in conditioning when approximately solving the Hamilton-Jacobi-Bellman equation to obtain optimal controls. We use an intuitive example to provide insights on the central excitation issue under discussion, and we demonstrate the effectiveness of the proposed procedure on a real-world hypersonic vehicle (HSV) application.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Direct Antenna Frequency-Hopped M-FSK Modulation With Time-Modulated Arrays
We present an innovative approach that simultaneously enables direct antenna frequency-hopped M-ary frequency shift keying (DAFH-MFSK) modulation and beamsteering through the use of time-modulated arrays (TMAs). The dist…
Continuous-Time Reinforcement Learning: New Design Algorithms with Theoretical Insights and Performance Guarantees
Continuous-time nonlinear optimal control problems hold great promise in real-world applications. After decades of development, reinforcement learning (RL) has achieved some of the greatest successes as a general nonline…
reinforcement-learningReinforcement Learning (RL)Reinforcement Learning with Feedback-modulated TD-STDP
Spiking neuron networks have been used successfully to solve simple reinforcement learning tasks with continuous action set applying learning rules based on spike-timing-dependent plasticity (STDP). However, most of thes…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Online Adaptive Optimal Control Algorithm Based on Synchronous Integral Reinforcement Learning With Explorations
In this paper, we present a novel algorithm named synchronous integral Q-learning, which is based on synchronous policy iteration, to solve the continuous-time infinite horizon optimal control problems of input-affine sy…
Q-Learningreinforcement-learningReinforcement Learning (RL)EquiFiLM: Charge-Conditioned Equivariant Force Fields via Feature-wise Linear Modulation
Foundation machine learning force fields (MLFFs) such as MACE-MP-0 and UMA cover broad chemical space at near density functional theory (DFT) accuracy. However, they assume equilibrium ground-state physics and do not nat…