Distributional Reinforcement Learning-based Energy Arbitrage Strategies in Imbalance Settlement Mechanism

2023-12-23 · Seyed Soroush Karimi Madahi, Bert Claessens, Chris Develder

Growth in the penetration of renewable energy sources makes supply more uncertain and leads to an increase in the system imbalance. This trend, together with the single imbalance pricing, opens an opportunity for balance responsible parties (BRPs) to perform energy arbitrage in the imbalance settlement mechanism. To this end, we propose a battery control framework based on distributional reinforcement learning (DRL). Our proposed control framework takes a risk-sensitive perspective, allowing BRPs to adjust their risk preferences: we aim to optimize a weighted sum of the arbitrage profit and a risk measure while constraining the daily number of cycles for the battery. We assess the performance of our proposed control framework using the Belgian imbalance prices of 2022 and compare two state-of-the-art RL methods, deep Q learning and soft actor-critic. Results reveal that the distributional soft actor-critic method can outperform other methods. Moreover, we note that our fully risk-averse agent appropriately learns to hedge against the risk related to the unknown imbalance price by (dis)charging the battery only when the agent is more certain about the price.

📄 PDF Abstract BibTeX arXiv:2401.00015

Code (0)

등록된 구현이 없습니다.

Tasks

Distributional Reinforcement LearningQ-Learningreinforcement-learning

Control Policy Correction Framework for Reinforcement Learning-based Energy Arbitrage Strategies

2024-04-29 · Seyed Soroush Karimi Madahi, Gargya Gokhale, Marie-Sophie Verwee, Bert Claessens 외

A continuous rise in the penetration of renewable energy sources, along with the use of the single imbalance pricing, provides a new opportunity for balance responsible parties to reduce their cost through energy arbitra…

Knowledge Distillation reinforcement-learning Reinforcement Learning (RL)

Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning

2021-06-10 · Daniel J. B. Harrold, Jun Cao, Zhong Fan

As the world seeks to become more sustainable, intelligent solutions are needed to increase the penetration of renewable energy. In this paper, the model-free deep reinforcement learning algorithm Rainbow Deep Q-Networks…

continuous-control Continuous Control Deep Reinforcement Learning reinforcement-learning+2

Energy Storage Arbitrage in Real-Time Markets via Reinforcement Learning

2017-11-08 · Hao Wang, Baosen Zhang

In this paper, we derive a temporal arbitrage policy for storage via reinforcement learning. Real-time price arbitrage is an important source of revenue for storage units, but designing good strategies have proven to be …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-market value-stacking: Battery control for combined imbalance participation and non-uniform FCR bidding

2026-05-12 · Celle Hendrickx, Fabio Pavirani, Chris Develder arxiv

The growing share of Renewable Energy Sources (RES) in modern power systems increases both grid imbalances and frequency deviations, reinforcing the need for ancillary services such as Frequency Containment Reserve (FCR)…

Reinforcement Learning

Interpretable Deep Reinforcement Learning for Optimizing Heterogeneous Energy Storage Systems

2023-10-20 · Luolin Xiong, Yang Tang, Chensheng Liu, Shuai Mao 외

Energy storage systems (ESS) are pivotal component in the energy market, serving as both energy suppliers and consumers. ESS operators can reap benefits from energy arbitrage by optimizing operations of storage equipment…

Decision Making Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1