Arbitrage of Energy Storage in Electricity Markets with Deep Reinforcement Learning
In this letter, we address the problem of controlling energy storage systems (ESSs) for arbitrage in real-time electricity markets under price uncertainty. We first formulate this problem as a Markov decision process, and then develop a deep reinforcement learning based algorithm to learn a stochastic control policy that maps a set of available information processed by a recurrent neural network to ESSs' charging/discharging actions. Finally, we verify the effectiveness of our algorithm using real-time electricity prices from PJM.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Transferable Energy Storage Bidder
Energy storage resources must consider both price uncertainties and their physical operating characteristics when participating in wholesale electricity markets. This is a challenging problem as electricity prices are hi…
Transfer LearningStrategic Storage Investment in Electricity Markets
Arbitrage is one important revenue source for energy storage in electricity markets. However, a large amount of storage in the market will impact the energy price and reduce potential revenues. This can lead to strategic…
On the Stability of Strategic Energy Storage Operation in Wholesale Electricity Markets (Extended Version)
High shares of variable renewable energy necessitate substantial energy storage capacity. However, it remains unclear how to design a market that, on the one hand, ensures a stable and sufficient income for storage firms…
A Decision-Focused Predict-then-Bid Framework for Strategic Energy Storage
This paper introduces a novel decision-focused framework for energy storage arbitrage bidding. Inspired by the bidding process for energy storage in electricity markets, we propose a predict-then-bid end-to-end method in…
Proximal Policy Optimization Based Reinforcement Learning for Joint Bidding in Energy and Frequency Regulation Markets
Driven by the global decarbonization effort, the rapid integration of renewable energy into the conventional electricity grid presents new challenges and opportunities for the battery energy storage system (BESS) partici…
Deep Reinforcement Learning