Online Reinforcement Learning Control by Direct Heuristic Dynamic Programming: from Time-Driven to Event-Driven
In this paper time-driven learning refers to the machine learning method that updates parameters in a prediction model continuously as new data arrives. Among existing approximate dynamic programming (ADP) and reinforcement learning (RL) algorithms, the direct heuristic dynamic programming (dHDP) has been shown an effective tool as demonstrated in solving several complex learning control problems. It continuously updates the control policy and the critic as system states continuously evolve. It is therefore desirable to prevent the time-driven dHDP from updating due to insignificant system event such as noise. Toward this goal, we propose a new event-driven dHDP. By constructing a Lyapunov function candidate, we prove the uniformly ultimately boundedness (UUB) of the system states and the weights in the critic and the control policy networks. Consequently we show the approximate control and cost-to-go function approaching Bellman optimality within a finite bound. We also illustrate how the event-driven dHDP algorithm works in comparison to the original time-driven dHDP.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement Learning (RL)Similar Papers 제목 키워드 기반
Toward Reliable Designs of Data-Driven Reinforcement Learning Tracking Control for Euler-Lagrange Systems
This paper addresses reinforcement learning based, direct signal tracking control with an objective of developing mathematically suitable and practically useful design approaches. Specifically, we aim to provide reliable…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Chasing Stability: Humanoid Running via Control Lyapunov Function Guided Reinforcement Learning
Achieving highly dynamic behaviors on humanoid robots, such as running, requires controllers that are both robust and precise, and hence difficult to design. Classical control methods offer valuable insight into how such…
Reinforcement LearningLinear convergence of a policy gradient method for some finite horizon continuous time control problems
Despite its popularity in the reinforcement learning community, a provably convergent policy gradient method for continuous space-time control problems with nonlinear state dynamics has been elusive. This paper proposes …
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Temperature as a Meta-Policy: Adaptive Temperature in LLM Reinforcement Learning
Temperature is a crucial hyperparameter in large language models (LLMs), controlling the trade-off between exploration and exploitation during text generation. High temperatures encourage diverse but noisy outputs, while…
Reinforcement LearningMathematical ReasoningText GenerationDeep Reinforcement Learning-Aided Frequency Control of LCC-S Resonant Converters for Wireless Power Transfer Systems
This paper presents a novel deep reinforcement learning (DRL)-based control strategy for achieving precise and robust output voltage regulation in LCC-S resonant converters, specifically designed for wireless power trans…
Deep Reinforcement Learning