Control Policy Correction Framework for Reinforcement Learning-based Energy Arbitrage Strategies

2024-04-29 · Seyed Soroush Karimi Madahi, Gargya Gokhale, Marie-Sophie Verwee, Bert Claessens, Chris Develder

A continuous rise in the penetration of renewable energy sources, along with the use of the single imbalance pricing, provides a new opportunity for balance responsible parties to reduce their cost through energy arbitrage in the imbalance settlement mechanism. Model-free reinforcement learning (RL) methods are an appropriate choice for solving the energy arbitrage problem due to their outstanding performance in solving complex stochastic sequential problems. However, RL is rarely deployed in real-world applications since its learned policy does not necessarily guarantee safety during the execution phase. In this paper, we propose a new RL-based control framework for batteries to obtain a safe energy arbitrage strategy in the imbalance settlement mechanism. In our proposed control framework, the agent initially aims to optimize the arbitrage revenue. Subsequently, in the post-processing step, we correct (constrain) the learned policy following a knowledge distillation process based on properties that follow human intuition. Our post-processing step is a generic method and is not restricted to the energy arbitrage domain. We use the Belgian imbalance price of 2023 to evaluate the performance of our proposed framework. Furthermore, we deploy our proposed control framework on a real battery to show its capability in the real world.

📄 PDF Abstract BibTeX arXiv:2404.18821

Code (0)

등록된 구현이 없습니다.

Tasks

Knowledge Distillation reinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Safe Deployment of Offline Reinforcement Learning via Input Convex Action Correction

2025-07-30 · Alex Durkin, Jasper Stolte, Matthew Jones, Raghuraman Pitchumani 외 arxiv

Offline reinforcement learning (offline RL) offers a promising framework for developing control strategies in chemical process systems using historical data, without the risks or costs of online experimentation. This wor…

Reinforcement Learning Offline RL

PhysQ: A Physics Informed Reinforcement Learning Framework for Building Control

2022-11-21 · Gargya Gokhale, Bert Claessens, Chris Develder

Large-scale integration of intermittent renewable energy sources calls for substantial demand side flexibility. Given that the built environment accounts for approximately 40% of total energy consumption in EU, unlocking…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Off-policy Reinforcement Learning with Optimistic Exploration and Distribution Correction

2021-10-22 · Jiachen Li, Shuo Cheng, Zhenyu Liao, Huayan Wang 외

Improving the sample efficiency of reinforcement learning algorithms requires effective exploration. Following the principle of $\textit{optimism in the face of uncertainty}$ (OFU), we train a separate exploration policy…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings

2026-06-01 · Hallah Shahid Butt, Qiong Huang, Gökhan Demirel, Kevin Förderer 외 arxiv

The increasing integration of renewable energy sources into power systems, particularly in buildings equipped with photovoltaic (PV) panels and energy storage systems, introduces significant complexity in energy systems.…

Reinforcement Learning

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

2019-01-15 · Guanyu Gao, Jie Li, Yonggang Wen

Heating, Ventilation, and Air Conditioning (HVAC) is extremely energy-consuming, accounting for 40% of total building energy consumption. Therefore, it is crucial to design some energy-efficient building thermal control …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)