Initial Distribution Sensitivity of Constrained Markov Decision Processes
Constrained Markov Decision Processes (CMDPs) are notably more complex to solve than standard MDPs due to the absence of universally optimal policies across all initial state distributions. This necessitates re-solving the CMDP whenever the initial distribution changes. In this work, we analyze how the optimal value of CMDPs varies with different initial distributions, deriving bounds on these variations using duality analysis of CMDPs and perturbation analysis in linear programming. Moreover, we show how such bounds can be used to analyze the regret of a given policy due to unknown variations of the initial distribution.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Constrained Active Classification Using Partially Observable Markov Decision Processes
In this work, we study the problem of actively classifying the attributes of dynamical systems characterized as a finite set of Markov decision process (MDP) models. We are interested in finding strategies that actively …
AttributeClassificationMedical DiagnosisSensitivity analysis for finite Markov chains in discrete time
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This is done by considering as basic uncertainty models the so-called …
SensitivitySuccessive Convex Approximation Based Off-Policy Optimization for Constrained Reinforcement Learning
We propose a successive convex approximation based off-policy optimization (SCAOPO) algorithm to solve the general constrained reinforcement learning problem, which is formulated as a constrained Markov decision process …
reinforcement-learningReinforcement Learning (RL)On Bellman's principle of optimality and Reinforcement learning for safety-constrained Markov decision process
We study optimality for the safety-constrained Markov decision process which is the underlying framework for safe reinforcement learning. Specifically, we consider a constrained Markov decision process (with finite state…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Learning Policies for Markov Decision Processes from Data
We consider the problem of learning a policy for a Markov decision process consistent with data captured on the state-actions pairs followed by the policy. We assume that the policy belongs to a class of parameterized po…
Robot Navigation