Verification of Dissipativity and Evaluation of Storage Function in Economic Nonlinear MPC using Q-Learning
In the Economic Nonlinear Model Predictive (ENMPC) context, closed-loop stability relates to the existence of a storage function satisfying a dissipation inequality. Finding the storage function is in general -- for nonlinear dynamics and cost -- challenging, and has attracted attentions recently. Q-Learning is a well-known Reinforcement Learning (RL) techniques that attempts to capture action-value functions based on the state-input transitions and stage cost of the system. In this paper, we present the use of the Q-Learning approach to obtain the storage function and verify the dissipativity for discrete-time systems subject to state-input constraints. We show that undiscounted Q-learning is able to capture the storage function for dissipative problems when the parameterization is rich enough. The efficiency of the proposed method will be illustrated in the different case studies.
Code (0)
등록된 구현이 없습니다.
Tasks
Q-LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Functional Stability of Discounted Markov Decision Processes Using Economic MPC Dissipativity Theory
This paper discusses the functional stability of closed-loop Markov Chains under optimal policies resulting from a discounted optimality criterion, forming Markov Decision Processes (MDPs). We investigate the stability o…
Model Predictive ControlQ-LearningvalidEconomic MPC of Markov Decision Processes: Dissipativity in Undiscounted Infinite-Horizon Optimal Control
Economic Model Predictive Control (MPC) dissipativity theory is central to discussing the stability of policies resulting from minimizing economic stage costs. In its current form, the dissipativity theory for economic M…
Model Predictive ControlOn the one-shot data-driven verification of dissipativity of LTI systems with general quadratic supply rate function
Based on a one-shot input-output set of data from an LTI system, we present a verification method of dissipativity property based on a general quadratic supply-rate function. We show the applicability of our approach for…
Linearly discounted economic MPC without terminal conditions for periodic optimal operation
In this work, we study economic model predictive control (MPC) in situations where the optimal operating behavior is periodic. In such a setting, the performance of a standard economic MPC scheme without terminal conditi…
Model Predictive ControlDissipativity in economic model predictive control: beyond steady-state optimality
This chapter provides a concise survey on different dissipativity conditions that have appeared in the literature on economic model predictive control and discusses their decisive role in this context.
Model Predictive ControlSurvey