Verification of Markov Decision Processes with Risk-Sensitive Measures
We develop a method for computing policies in Markov decision processes with risk-sensitive measures subject to temporal logic constraints. Specifically, we use a particular risk-sensitive measure from cumulative prospect theory, which has been previously adopted in psychology and economics. The nonlinear transformation of the probabilities and utility functions yields a nonlinear programming problem, which makes computation of optimal policies typically challenging. We show that this nonlinear weighting function can be accurately approximated by the difference of two convex functions. This observation enables efficient policy computation using convex-concave programming. We demonstrate the effectiveness of the approach on several scenarios.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Markov Decision Processes with Risk-Sensitive Criteria: An Overview
The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term 'risk-sensitive' refers here to the use of the Optimized Certainty Equivalent as a means to measure expe…
An Actor-Critic Algorithm with Function Approximation for Risk Sensitive Cost Markov Decision Processes
In this paper, we consider the risk-sensitive cost criterion with exponentiated costs for Markov decision processes and develop a model-free policy gradient algorithm in this setting. Unlike additive cost criteria such a…
Inverse Risk-Sensitive Reinforcement Learning
We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of mod…
Decision Makingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Robust Risk-Sensitive Reinforcement Learning with Conditional Value-at-Risk
Robust Markov Decision Processes (RMDPs) have received significant research interest, offering an alternative to standard Markov Decision Processes (MDPs) that often assume fixed transition probabilities. RMDPs address t…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Regret Bounds for Markov Decision Processes with Recursive Optimized Certainty Equivalents
The optimized certainty equivalent (OCE) is a family of risk measures that cover important examples such as entropic risk, conditional value-at-risk and mean-variance models. In this paper, we propose a new episodic risk…
reinforcement-learningReinforcement Learning (RL)