Reinforcement Learning of Markov Decision Processes with Peak Constraints

2019-01-23 · Ather Gattami

In this paper, we consider reinforcement learning of Markov Decision Processes (MDP) with peak constraints, where an agent chooses a policy to optimize an objective and at the same time satisfy additional constraints. The agent has to take actions based on the observed states, reward outputs, and constraint-outputs, without any knowledge about the dynamics, reward functions, and/or the knowledge of the constraint-functions. We introduce a game theoretic approach to construct reinforcement learning algorithms where the agent maximizes an unconstrained objective that depends on the simulated action of the minimizing opponent which acts on a finite set of actions and the output data of the constraint functions (rewards). We show that the policies obtained from maximin Q-learning converge to the optimal policies. To the best of our knowledge, this is the first time learning algorithms guarantee convergence to optimal stationary policies for the MDP problem with peak constraints for both discounted and expected average rewards.

📄 PDF Abstract BibTeX arXiv:1901.07839

Code (0)

등록된 구현이 없습니다.

Tasks

Q-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Art

2026-01-08 · Timofey Tomashevskiy arxiv

This work provides a state-of-the-art survey of continual safe online reinforcement learning (COSRL) methods. We discuss theoretical aspects, challenges, and open questions in building continual online safe reinforcement…

Reinforcement Learning Continual Learning

Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learning

2023-04-29 · Liangyu Zhang, Yang Peng, Wenhao Yang, Zhihua Zhang

We propose a novel generalization of constrained Markov decision processes (CMDPs) that we call the \emph{semi-infinitely constrained Markov decision process} (SICMDP). Particularly, we consider a continuum of constraint…

Decision Making Deep Reinforcement Learning Model-based Reinforcement Learning reinforcement-learning+2

Safe Reinforcement Learning for Constrained Markov Decision Processes with Stochastic Stopping Time

2024-03-23 · Abhijit Mazumdar, Rafal Wisniewski, Manuela L. Bujorianu

In this paper, we present an online reinforcement learning algorithm for constrained Markov decision processes with a safety constraint. Despite the necessary attention of the scientific community, considering stochastic…

Efficient Exploration Safe Reinforcement Learning

Safe Reinforcement Learning in Constrained Markov Decision Processes

2020-08-15 · ICML 2020 1 · Akifumi Wachi, Yanan Sui

Safe reinforcement learning has been a promising approach for optimizing the policy of an agent that operates in safety-critical applications. In this paper, we propose an algorithm, SNO-MDP, that explores and optimizes …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Reinforcement Learning

Markov Abstractions for PAC Reinforcement Learning in Non-Markov Decision Processes

2022-04-29 · Alessandro Ronca, Gabriel Paludo Licks, Giuseppe De Giacomo

Our work aims at developing reinforcement learning algorithms that do not rely on the Markov assumption. We consider the class of Non-Markov Decision Processes where histories can be abstracted into a finite set of state…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)