Policy Gradient for Coherent Risk Measures

2015-02-13 · NeurIPS 2015 12 · Aviv Tamar, Yin-Lam Chow, Mohammad Ghavamzadeh, Shie Mannor

Several authors have recently developed risk-sensitive policy gradient methods that augment the standard expected cost minimization problem with a measure of variability in cost. These studies have focused on specific risk-measures, such as the variance or conditional value at risk (CVaR). In this work, we extend the policy gradient method to the whole class of coherent risk measures, which is widely accepted in finance and operations research, among other fields. We consider both static and time-consistent dynamic risk measures. For static risk measures, our approach is in the spirit of policy gradient algorithms and combines a standard sampling approach with convex programming. For dynamic risk measures, our approach is actor-critic style and involves explicit approximation of value function. Most importantly, our contribution presents a unified approach to risk-sensitive reinforcement learning that generalizes and extends previous results.

📄 PDF Abstract BibTeX arXiv:1502.03919

Code (0)

등록된 구현이 없습니다.

Tasks

Policy Gradient MethodsReinforcement Learning

Policy Gradient Methods for Risk-Sensitive Distributional Reinforcement Learning with Provable Convergence

2024-05-23 · Minheng Xiao, Xian Yu, Lei Ying

Risk-sensitive reinforcement learning (RL) is crucial for maintaining reliable performance in high-stakes applications. While traditional RL methods aim to learn a point estimate of the random cumulative cost, distributi…

Distributional Reinforcement Learning Policy Gradient Methods Reinforcement Learning (RL)

Risk-Sensitive Reinforcement Learning via Policy Gradient Search

2018-10-22 · Prashanth L. A., Michael Fu

The objective in a traditional reinforcement learning (RL) problem is to find a policy that optimizes the expected value of a performance metric such as the infinite-horizon cumulative discounted or long-run average cost…

Policy Gradient Methods reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Bayesian Risk-Sensitive Policy Optimization For MDPs With General Loss Functions

2025-09-19 · Xiaoshuang Wang, Yifan Lin, Enlu Zhou arxiv

Motivated by many application problems, we consider Markov decision processes (MDPs) with a general loss function and unknown parameters. To mitigate the epistemic uncertainty associated with unknown parameters, we take …

A policy gradient approach for optimization of smooth risk measures

2022-02-22 · Nithia Vijayan, Prashanth L. A

We propose policy gradient algorithms for solving a risk-sensitive reinforcement learning (RL) problem in on-policy as well as off-policy settings. We consider episodic Markov decision processes, and model the risk using…

reinforcement-learning Reinforcement Learning (RL)

Deep Reinforcement Learning for Equal Risk Option Pricing and Hedging under Dynamic Expectile Risk Measures

2021-09-29 · Saeed Marzban, Erick Delage, Jonathan Li

Recently equal risk pricing, a framework for fair derivative pricing, was extended to consider coherent risk measures. However, all current implementations either employ a static risk measure or are based on traditional …

Deep Reinforcement Learning