Automatic Exploration Process Adjustment for Safe Reinforcement Learning with Joint Chance Constraint Satisfaction

2021-03-05 · Yoshihiro Okawa, Tomotake Sasaki, Hidenao Iwane

In reinforcement learning (RL) algorithms, exploratory control inputs are used during learning to acquire knowledge for decision making and control, while the true dynamics of a controlled object is unknown. However, this exploring property sometimes causes undesired situations by violating constraints regarding the state of the controlled object. In this paper, we propose an automatic exploration process adjustment method for safe RL in continuous state and action spaces utilizing a linear nominal model of the controlled object. Specifically, our proposed method automatically selects whether the exploratory input is used or not at each time depending on the state and its predicted value as well as adjusts the variance-covariance matrix used in the Gaussian policy for exploration. We also show that our exploration process adjustment method theoretically guarantees the satisfaction of the constraints with the pre-specified probability, that is, the satisfaction of a joint chance constraint at every time. Finally, we illustrate the validity and the effectiveness of our method through numerical simulation.

📄 PDF Abstract BibTeX arXiv:2103.03656

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingObjectreinforcement-learning Reinforcement Learning (RL)Safe Reinforcement Learning

Meta SAC-Lag: Towards Deployable Safe Reinforcement Learning via MetaGradient-based Hyperparameter Tuning

2024-08-15 · Homayoun Honari, Amir Mehdi Soufi Enayati, Mehran Ghafarian Tamizi, Homayoun Najjaran

Safe Reinforcement Learning (Safe RL) is one of the prevalently studied subcategories of trial-and-error-based methods with the intention to be deployed on real-world systems. In safe RL, the goal is to maximize reward p…

Safe Exploration Safe Reinforcement Learning

SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration

2026-06-08 · Kaustubh Mani, Yann Pequignot, Vincent Mai, Liam Paull arxiv

Safe exploration is a prerequisite for deploying reinforcement learning (RL) agents in safety-critical domains. In this paper, we approach safe exploration through the lens of epistemic uncertainty, where the actor's sen…

Reinforcement Learning

Safe Exploration of State and Action Spaces in Reinforcement Learning

2014-02-04 · Javier Garcia, Fernando Fernandez

In this paper, we consider the important problem of safe exploration in reinforcement learning. While reinforcement learning is well-suited to domains with complex transition dynamics and high-dimensional state-action sp…

Efficient Exploration Management reinforcement-learning Reinforcement Learning+2

Cautious Reinforcement Learning with Logical Constraints

2020-02-26 · Mohammadhosein Hasanbeig, Alessandro Abate, Daniel Kroening

This paper presents the concept of an adaptive safe padding that forces Reinforcement Learning (RL) to synthesise optimal control policies while ensuring safety during the learning process. Policies are synthesised to sa…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey

2024-04-22 · Maeva Guerrier, Hassan Fouad, Giovanni Beltrame

Reinforcement learning is a powerful technique for developing new robot behaviors. However, typical lack of safety guarantees constitutes a hurdle for its practical application on real robots. To address this issue, safe…

Lifelong learning reinforcement-learning Reinforcement Learning Safe Reinforcement Learning