Coordinated Frequency Control through Safe Reinforcement Learning
With widespread deployment of renewables, the electric power grids are experiencing increasing dynamics and uncertainties, with its secure operation being threatened. Existing frequency control schemes based on day-ahead offline analysis and minute-level online sensitivity calculations are difficult to adapt to rapidly changing system states. In particular, they are unable to facilitate coordinated control of system frequency and power flows. A refined approach and tools are urgently needed to assist system operators to make timely decisions. This paper proposes a novel model-free coordinated frequency control framework based on safe reinforcement learning, with multiple control objectives considered. The load frequency control problem is modeled as a constrained Markov decision process, which can be solved by an AI agent continuously interacting with the grid to achieve sub-second decision making. Extensive numerical experiments conducted at East China Power Grid demonstrate the effectiveness and promise of the proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
AI AgentDecision Makingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Safe Reinforcement LearningSimilar Papers 제목 키워드 기반
Data-driven Coordinated AC/DC Control Strategy for Frequency Safety
With high penetrations of renewable energy and power electronics converters, less predictable operating conditions and strong uncertainties in under-frequency events pose challenges for emergency frequency control (EFC).…
Coordinated Frequency Regulation in Grid-Forming Storage Network via Safety-Consensus
Inverter-based storages are poised to play a prominent role in future power grids with massive renewable generation. Grid-forming inverters (GFMs) are emerging as a dominant technology with synchronous generators (SG)-li…
Hierarchical Reinforcement Learning with Low-Level MPC for Multi-Agent Control
Achieving safe and coordinated behavior in dynamic, constraint-rich environments remains a major challenge for learning-based control. Pure end-to-end learning often suffers from poor sample efficiency and limited reliab…
Hierarchical Reinforcement LearningA Multi-Agent Deep Reinforcement Learning Method for Cooperative Load Frequency Control of a Multi-Area Power System
This paper proposes a data-driven cooperative method for load frequency control (LFC) of the multi-area power system based on multi-agent deep reinforcement learning (MA-DRL) in continuous action domain. The proposed met…
Deep Reinforcement LearningTowards Low-Gravity Planetary Exploration using Reinforcement Learning for Walking, Jumping, and In-flight Attitude Control
This paper presents reinforcement learning (RL) policies for dynamic quadrupedal locomotion in planetary exploration scenarios. Building on a taskoptimized quadruped with a 5-bar leg design, we develop RL policies for wa…
Reinforcement Learning