Computationally Efficient Safe Reinforcement Learning for Power Systems

2021-10-20 · Daniel Tabas, Baosen Zhang

We propose a computationally efficient approach to safe reinforcement learning (RL) for frequency regulation in power systems with high levels of variable renewable energy resources. The approach draws on set-theoretic control techniques to craft a neural network-based control policy that is guaranteed to satisfy safety-critical state constraints, without needing to solve a model predictive control or projection problem in real time. By exploiting the properties of robust controlled-invariant polytopes, we construct a novel, closed-form "safety-filter" that enables end-to-end safe learning using any policy gradient-based RL algorithm. We then apply the safety filter in conjunction with the deep deterministic policy gradient (DDPG) algorithm to regulate frequency in a modified 9-bus power system, and show that the learned policy is more cost-effective than robust linear feedback control techniques while maintaining the same safety guarantee. We also show that the proposed paradigm outperforms DDPG augmented with constraint violation penalties.

📄 PDF Abstract BibTeX arXiv:2110.10333

Code (0)

등록된 구현이 없습니다.

Tasks

Model Predictive Controlreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Reinforcement Learning

Methods 이 논문이 사용한 방법론

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Adam 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Batch Normalization 설명 없음

Weight Decay 설명 없음

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

DDPG DDPG, or Deep Deterministic Policy Gradient, is an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action…

A Critical Review of Safe Reinforcement Learning Techniques in Smart Grid Applications

2024-09-24 · Van-Hai Bui, Srijita Das, Akhtar Hussain, Guilherme Vieira Hollweg 외

The high penetration of distributed energy resources (DERs) in modern smart power systems introduces unforeseen uncertainties for the electricity sector, leading to increased complexity and difficulty in the operation an…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Safe Reinforcement Learning

A Review of Safe Reinforcement Learning Methods for Modern Power Systems

2024-06-29 · Tong Su, Tong Wu, Junbo Zhao, Anna Scaglione 외

Given the availability of more comprehensive measurement data in modern power systems, reinforcement learning (RL) has gained significant interest in operation and control. Conventional RL relies on trial-and-error inter…

energy management Reinforcement Learning (RL)Safe Reinforcement Learning Scheduling

Hierarchical Reinforcement Learning with Runtime Safety Shielding for Power Grid Operation

2026-04-15 · Gitesh Malik arxiv

Reinforcement learning has shown promise for automating power-grid operation tasks such as topology control and congestion management. However, its deployment in real-world power systems remains limited by strict safety …

Hierarchical Reinforcement Learning Zero-shot Generalization

Barrier Function-based Safe Reinforcement Learning for Emergency Control of Power Systems

2021-03-26 · Thanh Long Vu, Sayak Mukherjee, Renke Huang, Qiuhua Hung

Under voltage load shedding has been considered as a standard and effective measure to recover the voltage stability of the electric power grid under emergency and severe conditions. However, this scheme usually trips a …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Safe Reinforcement Learning

Efficient Learning of Voltage Control Strategies via Model-based Deep Reinforcement Learning

2022-12-06 · Ramij R. Hossain, Tianzhixi Yin, Yan Du, Renke Huang 외

This article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results in mo…

Deep Reinforcement Learning Imitation Learning reinforcement-learning Reinforcement Learning (RL)