Network Topology Optimization via Deep Reinforcement Learning
Topology impacts important network performance metrics, including link utilization, throughput and latency, and is of central importance to network operators. However, due to the combinatorial nature of network topology, it is extremely difficult to obtain an optimal solution, especially since topology planning in networks also often comes with management-specific constraints. As a result, local optimization with hand-tuned heuristic methods from human experts are often adopted in practice. Yet, heuristic methods cannot cover the global topology design space while taking into account constraints, and cannot guarantee to find good solutions. In this paper, we propose a novel deep reinforcement learning (DRL) algorithm, called Advantage Actor Critic-Graph Searching (A2C-GS), for network topology optimization. A2C-GS consists of three novel components, including a verifier to validate the correctness of a generated network topology, a graph neural network (GNN) to efficiently approximate topology rating, and a DRL actor layer to conduct a topology search. A2C-GS can efficiently search over large topology space and output topology with satisfying performance. We conduct a case study based on a real network scenario, and our experimental results demonstrate the superior performance of A2C-GS in terms of both efficiency and performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningGraph Neural NetworkManagementreinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Generative Design by Reinforcement Learning: Enhancing the Diversity of Topology Optimization Designs
Generative design refers to computational design methods that can automatically conduct design exploration under constraints defined by designers. Among many approaches, topology optimization-based generative designs aim…
CPUDiversityGPUreinforcement-learning+1Accelerated DC loadflow solver for topology optimization
We present a massively parallel solver that accelerates DC loadflow computations for power grid topology optimization tasks. Our approach leverages low-rank updates of the Power Transfer Distribution Factors (PTDFs) to r…
DiversityHUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach
With the growth of Renewable Energy (RE) generation, the operation of power grids has become increasingly complex. One solution could be automated grid operation, where Deep Reinforcement Learning (DRL) has repeatedly sh…
Deep Reinforcement LearningMulti-Objective Reinforcement Learning for Power Grid Topology Control
Transmission grid congestion increases as the electrification of various sectors requires transmitting more power. Topology control, through substation reconfiguration, can reduce congestion but its potential remains und…
Multi-Objective Reinforcement Learningreinforcement-learningReinforcement LearningCurriculum Based Reinforcement Learning of Grid Topology Controllers to Prevent Thermal Cascading
This paper describes how domain knowledge of power system operators can be integrated into reinforcement learning (RL) frameworks to effectively learn agents that control the grid's topology to prevent thermal cascading.…
reinforcement-learningReinforcement Learning (RL)RTE