A Safe Reinforcement Learning Algorithm for Supervisory Control of Power Plants

2024-01-23 · Yixuan Sun, Sami Khairy, Richard B. Vilim, Rui Hu, Akshay J. Dave

Traditional control theory-based methods require tailored engineering for each system and constant fine-tuning. In power plant control, one often needs to obtain a precise representation of the system dynamics and carefully design the control scheme accordingly. Model-free Reinforcement learning (RL) has emerged as a promising solution for control tasks due to its ability to learn from trial-and-error interactions with the environment. It eliminates the need for explicitly modeling the environment's dynamics, which is potentially inaccurate. However, the direct imposition of state constraints in power plant control raises challenges for standard RL methods. To address this, we propose a chance-constrained RL algorithm based on Proximal Policy Optimization for supervisory control. Our method employs Lagrangian relaxation to convert the constrained optimization problem into an unconstrained objective, where trainable Lagrange multipliers enforce the state constraints. Our approach achieves the smallest distance of violation and violation rate in a load-follow maneuver for an advanced Nuclear Power Plant design.

📄 PDF Abstract BibTeX arXiv:2401.13020

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning (RL)Safe Reinforcement Learning

Safe Control and Learning Using the Generalized Action Governor

2022-11-22 · Nan Li, Yutong Li, Ilya Kolmanovsky, Anouck Girard 외

This article introduces a general framework for safe control and learning based on the generalized action governor (AG). The AG is a supervisory scheme for augmenting a nominal closed-loop system with the ability of stri…

reinforcement-learning Reinforcement Learning (RL)

Reinforcement Learning for Task Specifications with Action-Constraints

2022-01-02 · Arun Raman, Keerthan Shagrithaya, Shalabh Bhatnagar

In this paper, we use concepts from supervisory control theory of discrete event systems to propose a method to learn optimal control policies for a finite-state Markov Decision Process (MDP) in which (only) certain sequ…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cages

2021-03-17 · Sampo Kuutti, Richard Bowden, Saber Fallah

The use of neural networks and reinforcement learning has become increasingly popular in autonomous vehicle control. However, the opaqueness of the resulting control policies presents a significant barrier to deploying n…

Autonomous Vehicles reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Critical Review of Safe Reinforcement Learning Techniques in Smart Grid Applications

2024-09-24 · Van-Hai Bui, Srijita Das, Akhtar Hussain, Guilherme Vieira Hollweg 외

The high penetration of distributed energy resources (DERs) in modern smart power systems introduces unforeseen uncertainties for the electricity sector, leading to increased complexity and difficulty in the operation an…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Safe Reinforcement Learning

Safety and Liveness Guarantees through Reach-Avoid Reinforcement Learning

2021-12-23 · Kai-Chieh Hsu, Vicenç Rubies-Royo, Claire J. Tomlin, Jaime F. Fisac

Reach-avoid optimal control problems, in which the system must reach certain goal conditions while staying clear of unacceptable failure modes, are central to safety and liveness assurance for autonomous robotic systems,…

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1