Safe deep reinforcement learning-based constrained optimal control scheme for active distribution networks
Reinforcement learning-based schemes are being recently applied for model-free voltage control in active distribution networks. However, existing reinforcement learning methods face challenges when it comes to continuous state and action spaces problems or problems with operation constraints. To address these limitations, this paper proposes an optimal voltage control scheme based on the safe deep reinforcement learning. In this scheme, the optimal voltage control problem is formulated as a constrained Markov decision process, in which both state and action spaces are continuous. To solve this problem efficiently, the deep deterministic policy gradient algorithm is utilized to learn the reactive power control policies, which determine the optimal control actions from the states. In contrast to existing reinforcement learning methods, deep deterministic policy gradient is naturally capable of addressing control problems with continuous state and action spaces. This is due to the utilization of deep neural networks to approximate both value function and policy. In addition, in order to handle the operation constraints in active distribution networks, a safe exploration approach is proposed to form a safety layer, which is composed directly on top the deep deterministic policy gradient actor network. This safety layer predicts the change in the constrained states and prevents the violation of active distribution networks operation constraints. Numerical simulations on modified IEEE test systems demonstrate that the proposed scheme successfully maintains all bus voltage within the allowed range, and reduces the system loss by 15% compared to the no control case.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Safe ExplorationSimilar Papers 제목 키워드 기반
On Bellman's principle of optimality and Reinforcement learning for safety-constrained Markov decision process
We study optimality for the safety-constrained Markov decision process which is the underlying framework for safe reinforcement learning. Specifically, we consider a constrained Markov decision process (with finite state…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Robust Safe Reinforcement Learning under Adversarial Disturbances
Safety is a primary concern when applying reinforcement learning to real-world control tasks, especially in the presence of external disturbances. However, existing safe reinforcement learning algorithms rarely account f…
reinforcement-learningReinforcement LearningSafe Reinforcement LearningFoundations of Safe Online Reinforcement Learning in the Linear Quadratic Regulator: $\sqrt{T}$-Regret
Understanding how to efficiently learn while adhering to safety constraints is essential for using online reinforcement learning in practical applications. However, proving rigorous regret bounds for safety-constrained r…
reinforcement-learningReinforcement LearningSafe Model-Based Reinforcement Learning for Systems with Parametric Uncertainties
Reinforcement learning has been established over the past decade as an effective tool to find optimal control policies for dynamical systems, with recent focus on approaches that guarantee safety during the learning and/…
Model-based Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Probabilistically safe and efficient model-based Reinforcement Learning
This paper proposes tackling safety-critical stochastic Reinforcement Learning (RL) tasks with a samplebased, model-based approach. At the core of the method lies a Model Predictive Control (MPC) scheme that acts as func…
Model-based Reinforcement LearningModel Predictive Controlreinforcement-learningReinforcement Learning+1