Proximal Policy Optimization with Graph Neural Networks for Optimal Power Flow

2022-12-23 · Ángela López-Cardona, Guillermo Bernárdez, Pere Barlet-Ros, Albert Cabellos-Aparicio

Optimal Power Flow (OPF) is a very traditional research area within the power systems field that seeks for the optimal operation point of electric power plants, and which needs to be solved every few minutes in real-world scenarios. However, due to the nonconvexities that arise in power generation systems, there is not yet a fast, robust solution technique for the full Alternating Current Optimal Power Flow (ACOPF). In the last decades, power grids have evolved into a typical dynamic, non-linear and large-scale control system, known as the power system, so searching for better and faster ACOPF solutions is becoming crucial. Appearance of Graph Neural Networks (GNN) has allowed the natural use of Machine Learning (ML) algorithms on graph data, such as power networks. On the other hand, Deep Reinforcement Learning (DRL) is known for its powerful capability to solve complex decision-making problems. Although solutions that use these two methods separately are beginning to appear in the literature, none has yet combined the advantages of both. We propose a novel architecture based on the Proximal Policy Optimization algorithm with Graph Neural Networks to solve the Optimal Power Flow. The objective is to design an architecture that learns how to solve the optimization problem and that is at the same time able to generalize to unseen scenarios. We compare our solution with the DCOPF in terms of cost after having trained our DRL agent on IEEE 30 bus system and then computing the OPF on that base network with topology changes

📄 PDF Abstract BibTeX arXiv:2212.12470

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingDeep Reinforcement Learning

Methods 이 논문이 사용한 방법론

None 설명 없음

BASE 설명 없음

Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policy

2019-06-25 · Boyi Liu, Qi Cai, Zhuoran Yang, Zhaoran Wang

Proximal policy optimization and trust region policy optimization (PPO and TRPO) with actor and critic parametrized by neural networks achieve significant empirical success in deep reinforcement learning. However, due to…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Neural Trust Region/Proximal Policy Optimization Attains Globally Optimal Policy

2019-12-01 · NeurIPS 2019 12 · Boyi Liu, Qi Cai, Zhuoran Yang, Zhaoran Wang

Deep Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)

ERPPO: Entropy Regularization-based Proximal Policy Optimization

2026-05-13 · Changha Lee, Gyusang Cho arxiv

Multi-Agent Proximal Policy Optimization (MAPPO) is a variant of the Proximal Policy Optimization (PPO) algorithm, specifically tailored for multi-agent reinforcement learning (MARL). MAPPO optimizes cooperative multi-ag…

Multi-agent Reinforcement Learning Object Localization Object Detection

Bridging Physics-Informed Neural Networks with Reinforcement Learning: Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO)

2023-02-01 · Amartya Mukherjee, Jun Liu

This paper introduces the Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO) algorithm into reinforcement learning. The Hamilton-Jacobi-Bellman (HJB) equation is used in control theory to evaluate the optimali…

MuJoCo reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

SUPERVISED POLICY UPDATE

2019-05-01 · ICLR 2019 5 · Quan Vuong, Yiming Zhang, Keith W. Ross

We propose a new sample-efficient methodology, called Supervised Policy Update (SPU), for deep reinforcement learning. Starting with data generated by the current policy, SPU formulates and solves a constrained optimizat…

Deep Reinforcement Learning MuJoCo Reinforcement Learning Reinforcement Learning (RL)