Scalable Volt-VAR Optimization using RLlib-IMPALA Framework: A Reinforcement Learning Approach

2024-02-24 · Alaa Selim, Yanzhu Ye, Junbo Zhao, Bo Yang

In the rapidly evolving domain of electrical power systems, the Volt-VAR optimization (VVO) is increasingly critical, especially with the burgeoning integration of renewable energy sources. Traditional approaches to learning-based VVO in expansive and dynamically changing power systems are often hindered by computational complexities. To address this challenge, our research presents a novel framework that harnesses the potential of Deep Reinforcement Learning (DRL), specifically utilizing the Importance Weighted Actor-Learner Architecture (IMPALA) algorithm, executed on the RAY platform. This framework, built upon RLlib-an industry-standard in Reinforcement Learning-ingeniously capitalizes on the distributed computing capabilities and advanced hyperparameter tuning offered by RAY. This design significantly expedites the exploration and exploitation phases in the VVO solution space. Our empirical results demonstrate that our approach not only surpasses existing DRL methods in achieving superior reward outcomes but also manifests a remarkable tenfold reduction in computational requirements. The integration of our DRL agent with the RAY platform facilitates the creation of RLlib-IMPALA, a novel framework that efficiently uses RAY's resources to improve system adaptability and control. RLlib-IMPALA leverages RAY's toolkit to enhance analytical capabilities and significantly speeds up training to become more than 10 times faster than other state-of-the-art DRL methods.

📄 PDF Abstract BibTeX arXiv:2402.15932

Code (1)

alaaselim-sudo/-rllib-impala 공식 구현

Tasks

Deep Reinforcement LearningDistributed Computingreinforcement-learning Reinforcement Learningrllib

RLlib: Abstractions for Distributed Reinforcement Learning

2017-12-26 · ICML 2018 7 · Eric Liang, Richard Liaw, Philipp Moritz, Robert Nishihara 외

Reinforcement learning (RL) algorithms involve the deep nesting of highly irregular computation patterns, each of which typically exhibits opportunities for distributed computation. We argue for distributing RL component…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)rllib

MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library

2022-10-11 · Siyi Hu, Yifan Zhong, Minquan Gao, Weixun Wang 외

A significant challenge facing researchers in the area of multi-agent reinforcement learning (MARL) pertains to the identification of a library that can offer fast and compatible development for multi-agent tasks and alg…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

IMPACT: Importance Weighted Asynchronous Architectures with Clipped Target Networks

2019-11-30 · ICLR 2020 1 · Michael Luo, Jiahao Yao, Richard Liaw, Eric Liang 외

The practical usage of reinforcement learning agents is often bottlenecked by the duration of training time. To accelerate training, practitioners often turn to distributed reinforcement learning architectures to paralle…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

2018-02-05 · ICML 2018 7 · Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan 외

In this work we aim to solve a large collection of tasks using a single reinforcement learning agent with a single set of parameters. A key challenge is to handle the increased amount of data and extended training time. …

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

RLlib Flow: Distributed Reinforcement Learning is a Dataflow Problem

2020-11-25 · NeurIPS 2021 12 · Eric Liang, Zhanghao Wu, Michael Luo, Sven Mika 외

Researchers and practitioners in the field of reinforcement learning (RL) frequently leverage parallel computation, which has led to a plethora of new algorithms and systems in the last few years. In this paper, we re-ex…