Control invariant set enhanced safe reinforcement learning: improved sampling efficiency, guaranteed stability and robustness
Reinforcement learning (RL) is an area of significant research interest, and safe RL in particular is attracting attention due to its ability to handle safety-driven constraints that are crucial for real-world applications. This work proposes a novel approach to RL training, called control invariant set (CIS) enhanced RL, which leverages the advantages of utilizing the explicit form of CIS to improve stability guarantees and sampling efficiency. Furthermore, the robustness of the proposed approach is investigated in the presence of uncertainty. The approach consists of two learning stages: offline and online. In the offline stage, CIS is incorporated into the reward design, initial state sampling, and state reset procedures. This incorporation of CIS facilitates improved sampling efficiency during the offline training process. In the online stage, RL is retrained whenever the predicted next step state is outside of the CIS, which serves as a stability criterion, by introducing a Safety Supervisor to examine the safety of the action and make necessary corrections. The stability analysis is conducted for both cases, with and without uncertainty. To evaluate the proposed approach, we apply it to a simulated chemical reactor. The results show a significant improvement in sampling efficiency during offline training and closed-loop stability guarantee in the online implementation, with and without uncertainty.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement Learning (RL)Safe Reinforcement LearningSimilar Papers 제목 키워드 기반
Control invariant set enhanced reinforcement learning for process control: improved sampling efficiency and guaranteed stability
Reinforcement learning (RL) is an area of significant research interest, and safe RL in particular is attracting attention due to its ability to handle safety-driven constraints that are crucial for real-world applicatio…
Reinforcement Learning (RL)Safe Exploration in Reinforcement Learning: Training Backup Control Barrier Functions with Zero Training Time Safety Violations
This paper introduces the reinforcement learning backup shield (RLBUS), an algorithm that guarantees safe exploration in reinforcement learning (RL) by incorporating backup control barrier functions (BCBFs). RLBUS constr…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Safe Exploration+1Robust Safe Reinforcement Learning under Adversarial Disturbances
Safety is a primary concern when applying reinforcement learning to real-world control tasks, especially in the presence of external disturbances. However, existing safe reinforcement learning algorithms rarely account f…
reinforcement-learningReinforcement LearningSafe Reinforcement LearningProvably Safe, Yet Scalable Reinforcement Learning
Safe reinforcement learning (RL) aims to learn policies that optimize rewards while satisfying constraints. Predominant approaches rely on soft-constrained policy optimization, which has achieved empirical success but do…
Reinforcement LearningEnergy-Efficient Control Adaptation with Safety Guarantees for Learning-Enabled Cyber-Physical Systems
Neural networks have been increasingly applied for control in learning-enabled cyber-physical systems (LE-CPSs) and demonstrated great promises in improving system performance and efficiency, as well as reducing the need…
Deep Reinforcement Learning