A Survey of Constraint Formulations in Safe Reinforcement Learning

2024-02-03 · Akifumi Wachi, Xun Shen, Yanan Sui

Safety is critical when applying reinforcement learning (RL) to real-world problems. As a result, safe RL has emerged as a fundamental and powerful paradigm for optimizing an agent's policy while incorporating notions of safety. A prevalent safe RL approach is based on a constrained criterion, which seeks to maximize the expected cumulative reward subject to specific safety constraints. Despite recent effort to enhance safety in RL, a systematic understanding of the field remains difficult. This challenge stems from the diversity of constraint representations and little exploration of their interrelations. To bridge this knowledge gap, we present a comprehensive review of representative constraint formulations, along with a curated selection of algorithms designed specifically for each formulation. In addition, we elucidate the theoretical underpinnings that reveal the mathematical mutual relations among common problem formulations. We conclude with a discussion of the current state and future directions of safe reinforcement learning research.

📄 PDF Abstract BibTeX arXiv:2402.02025

Code (0)

등록된 구현이 없습니다.

Tasks

Diversityreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Reinforcement LearningSurvey

Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability

2022-09-16 · Mengdi Xu, Zuxin Liu, Peide Huang, Wenhao Ding 외

A trustworthy reinforcement learning algorithm should be competent in solving challenging real-world problems, including {robustly} handling uncertainties, satisfying {safety} constraints to avoid catastrophic failures, …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Art

2026-01-08 · Timofey Tomashevskiy arxiv

This work provides a state-of-the-art survey of continual safe online reinforcement learning (COSRL) methods. We discuss theoretical aspects, challenges, and open questions in building continual online safe reinforcement…

Reinforcement Learning Continual Learning

A Survey on Reinforcement Learning in Aviation Applications

2022-11-03 · Pouria Razzaghi, Amin Tabrizian, Wei Guo, Shulu Chen 외

Compared with model-based control and optimization methods, reinforcement learning (RL) provides a data-driven, learning-based framework to formulate and solve sequential decision-making problems. The RL framework has be…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+2

State-wise Safe Reinforcement Learning: A Survey

2023-02-06 · WeiYe Zhao, Tairan He, Rui Chen, Tianhao Wei 외

Despite the tremendous success of Reinforcement Learning (RL) algorithms in simulation environments, applying RL to real-world applications still faces many challenges. A major concern is safety, in another word, constra…

Autonomous Driving reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+3

Enhancing Feasibility and Safety of Nonlinear Model Predictive Control with Discrete-Time Control Barrier Functions

2021-05-21 · Jun Zeng, Zhongyu Li, Koushil Sreenath

Safety is one of the fundamental problems in robotics. Recently, one-step or multi-step optimal control problems for discrete-time nonlinear dynamical system were formulated to offer tracking stability using control Lyap…

Model Predictive Control