Policy Gradients for Probabilistic Constrained Reinforcement Learning
This paper considers the problem of learning safe policies in the context of reinforcement learning (RL). In particular, we consider the notion of probabilistic safety. This is, we aim to design policies that maintain the state of the system in a safe set with high probability. This notion differs from cumulative constraints often considered in the literature. The challenge of working with probabilistic safety is the lack of expressions for their gradients. Indeed, policy optimization algorithms rely on gradients of the objective function and the constraints. To the best of our knowledge, this work is the first one providing such explicit gradient expressions for probabilistic constraints. It is worth noting that the gradient of this family of constraints can be applied to various policy-based algorithms. We demonstrate empirically that it is possible to handle probabilistic constraints in a continuous navigation problem.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Probabilistic Constraint for Safety-Critical Reinforcement Learning
In this paper, we consider the problem of learning safe policies for probabilistic-constrained reinforcement learning (RL). Specifically, a safe policy or controller is one that, with high probability, maintains the traj…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Safe Policies for Reinforcement Learning via Primal-Dual Methods
In this paper, we study the learning of safe policies in the setting of reinforcement learning problems. This is, we aim to control a Markov Decision Process (MDP) of which we do not know the transition probabilities, bu…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Anytime Safe Reinforcement Learning
This paper considers the problem of solving constrained reinforcement learning problems with anytime guarantees, meaning that the algorithmic solution returns a safe policy regardless of when it is terminated. Drawing in…
reinforcement-learningReinforcement LearningSafe Reinforcement LearningPIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos
Previously, the exploding gradient problem has been explained to be central in deep learning and model-based reinforcement learning, because it causes numerical issues and instability in optimization. Our experiments in …
Model-based Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Stabilizing Off-Policy Reinforcement Learning with Conservative Policy Gradients
In recent years, advances in deep learning have enabled the application of reinforcement learning algorithms in complex domains. However, they lack the theoretical guarantees which are present in the tabular setting and …
Deep Reinforcement LearningMuJoCoreinforcement-learningReinforcement Learning+1