Mixed-Integer Optimal Control via Reinforcement Learning: A Case Study on Hybrid Electric Vehicle Energy Management

2023-05-02 · Jinming Xu, Nasser Lashgarian Azad, Yuan Lin

Many optimal control problems require the simultaneous output of discrete and continuous control variables. These problems are usually formulated as mixed-integer optimal control (MIOC) problems, which are challenging to solve due to the complexity of the solution space. Numerical methods such as branch-and-bound are computationally expensive and undesirable for real-time control. This paper proposes a novel hybrid-action reinforcement learning (HARL) algorithm, twin delayed deep deterministic actor-Q (TD3AQ), for MIOC problems. TD3AQ combines the advantages of both actor-critic and Q-learning methods, and can handle the discrete and continuous action spaces simultaneously. The proposed algorithm is evaluated on a plug-in hybrid electric vehicle (PHEV) energy management problem, where real-time control of the discrete variables, clutch engagement/disengagement and gear shift, and continuous variable, engine torque, is essential to maximize fuel economy while satisfying driving constraints. Simulation outcomes demonstrate that TD3AQ achieves control results close to optimality when compared with dynamic programming (DP), with just 4.69% difference. Furthermore, it surpasses the performance of baseline reinforcement learning algorithms.

📄 PDF Abstract BibTeX arXiv:2305.01461

Code (1)

xujinming01/td3aq 공식 구현 pytorch

Tasks

continuous-control Continuous Controlenergy managementManagementQ-Learningreinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Integrating Reinforcement Learning and Model Predictive Control with Applications to Microgrids

2024-09-17 · Caio Fabio Oliveira da Silva, Azita Dabiri, Bart De Schutter

This work proposes an approach that integrates reinforcement learning and model predictive control (MPC) to solve finite-horizon optimal control problems in mixed-logical dynamical systems efficiently. Optimization-based…

Model Predictive Control reinforcement-learning Reinforcement Learning

Machine Learning Algorithms for Improving Exact Classical Solvers in Mixed Integer Continuous Optimization

2025-08-09 · Morteza Kimiaei, Vyacheslav Kungurtsev, Brian Olimba arxiv

Integer and mixed-integer nonlinear programming (INLP, MINLP) are central to logistics, energy, and scheduling, but remain computationally challenging. This survey examines how machine learning and reinforcement learning…

Reinforcement Learning

Integrating process design and control using reinforcement learning

2021-08-11 · Steven Sachio, Max Mowbray, Maria Papathanasiou, Ehecatl Antonio del Rio-Chanona 외

To create efficient-high performing processes, one must find an optimal design with its corresponding controller that ensures optimal operation in the presence of uncertainty. When comparing different process designs, fo…

Bilevel Optimization reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Fast Switching in Mixed-Integer Model Predictive Control

2024-11-28 · Artemi Makarow, Christian Kirches

We derive stability results for finite control set and mixed-integer model predictive control with a downstream oversampling phase. The presentation rests upon the inherent robustness of model predictive control with sta…

Model Predictive Control

Learning the cost-to-go for mixed-integer nonlinear model predictive control

2024-01-23 · Christopher A. Orrico, W. P. M. H. Heemels, Dinesh Krishnamoorthy

Application of nonlinear model predictive control (NMPC) to problems with hybrid dynamical systems, disjoint constraints, or discrete controls often results in mixed-integer formulations with both continuous and discrete…

Model Predictive Control Prediction