Curriculum Based Reinforcement Learning of Grid Topology Controllers to Prevent Thermal Cascading

2021-12-18 · Amarsagar Reddy Ramapuram Matavalam, Kishan Prudhvi Guddanti, Yang Weng, Venkataramana Ajjarapu

This paper describes how domain knowledge of power system operators can be integrated into reinforcement learning (RL) frameworks to effectively learn agents that control the grid's topology to prevent thermal cascading. Typical RL-based topology controllers fail to perform well due to the large search/optimization space. Here, we propose an actor-critic-based agent to address the problem's combinatorial nature and train the agent using the RL environment developed by RTE, the French TSO. To address the challenge of the large optimization space, a curriculum-based approach with reward tuning is incorporated into the training procedure by modifying the environment using network physics for enhanced agent learning. Further, a parallel training approach on multiple scenarios is employed to avoid biasing the agent to a few scenarios and make it robust to the natural variability in grid operations. Without these modifications to the training procedure, the RL agent failed for most test scenarios, illustrating the importance of properly integrating domain knowledge of physical systems for real-world RL learning. The agent was tested by RTE for the 2019 learning to run the power network challenge and was awarded the 2nd place in accuracy and 1st place in speed. The developed code is open-sourced for public use.

📄 PDF Abstract BibTeX arXiv:2112.09996

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning (RL)RTE

Curriculum-based Reinforcement Learning for Distribution System Critical Load Restoration

2022-03-08 · Xiangyu Zhang, Abinet Tesfaye Eseye, Bernard Knueven, Weijia Liu 외

This paper focuses on the critical load restoration problem in distribution systems following major outages. To provide fast online response and optimal sequential decision-making support, a reinforcement learning (RL) b…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Learning to run a power network challenge for training topology controllers

2019-12-05 · Antoine Marot, Benjamin Donnot, Camilo Romero, Luca Veyrin-Forrer 외

For power grid operations, a large body of research focuses on using generation redispatching, load shedding or demand side management flexibilities. However, a less costly and potentially more flexible option would be g…

Management Reinforcement Learning RTE

HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach

2024-05-01 · Malte Lehna, Clara Holzhüter, Sven Tomforde, Christoph Scholz

With the growth of Renewable Energy (RE) generation, the operation of power grids has become increasingly complex. One solution could be automated grid operation, where Deep Reinforcement Learning (DRL) has repeatedly sh…

Deep Reinforcement Learning

Hierarchical Reinforcement Learning with Runtime Safety Shielding for Power Grid Operation

2026-04-15 · Gitesh Malik arxiv

Reinforcement learning has shown promise for automating power-grid operation tasks such as topology control and congestion management. However, its deployment in real-world power systems remains limited by strict safety …

Hierarchical Reinforcement Learning Zero-shot Generalization

Multi-Objective Reinforcement Learning for Power Grid Topology Control

2025-01-27 · Thomas Lautenbacher, Ali Rajaei, Davide Barbieri, Jan Viebahn 외

Transmission grid congestion increases as the electrification of various sectors requires transmitting more power. Topology control, through substation reconfiguration, can reduce congestion but its potential remains und…

Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning