Policy Gradient Methods for Distortion Risk Measures
We propose policy gradient algorithms which learn risk-sensitive policies in a reinforcement learning (RL) framework. Our proposed algorithms maximize the distortion risk measure (DRM) of the cumulative reward in an episodic Markov decision process in on-policy and off-policy RL settings, respectively. We derive a variant of the policy gradient theorem that caters to the DRM objective, and integrate it with a likelihood ratio-based gradient estimation scheme. We derive non-asymptotic bounds that establish the convergence of our proposed algorithms to an approximate stationary point of the DRM objective.
Code (0)
등록된 구현이 없습니다.
Tasks
Policy Gradient Methodsreinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
A policy gradient approach for optimization of smooth risk measures
We propose policy gradient algorithms for solving a risk-sensitive reinforcement learning (RL) problem in on-policy as well as off-policy settings. We consider episodic Markov decision processes, and model the risk using…
reinforcement-learningReinforcement Learning (RL)Robust Reinforcement Learning with Dynamic Distortion Risk Measures
In a reinforcement learning (RL) setting, the agent's optimal strategy heavily depends on her risk preferences and the underlying model dynamics of the training environment. These two aspects influence the agent's abilit…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Policy Newton methods for Distortion Riskmetrics
We consider the problem of risk-sensitive control in a reinforcement learning (RL) framework. In particular, we aim to find a risk-optimal policy by maximizing the distortion riskmetric (DRM) of the discounted reward in …
Reinforcement LearningPolicy Gradient for Coherent Risk Measures
Several authors have recently developed risk-sensitive policy gradient methods that augment the standard expected cost minimization problem with a measure of variability in cost. These studies have focused on specific ri…
Policy Gradient MethodsReinforcement LearningStochastic Approximation Methods for Distortion Risk Measure Optimization
Distortion Risk Measures (DRMs) capture risk preferences in decision-making and serve as general criteria for managing uncertainty. This paper proposes gradient descent algorithms for DRM optimization based on two dual r…
Reinforcement LearningDensity Estimation