Optimizing the CVaR via Sampling

2014-04-15 · Aviv Tamar, Yonatan Glassner, Shie Mannor

Conditional Value at Risk (CVaR) is a prominent risk measure that is being used extensively in various domains. We develop a new formula for the gradient of the CVaR in the form of a conditional expectation. Based on this formula, we propose a novel sampling-based estimator for the CVaR gradient, in the spirit of the likelihood-ratio method. We analyze the bias of the estimator, and prove the convergence of a corresponding stochastic gradient descent algorithm to a local CVaR optimum. Our method allows to consider CVaR optimization in new domains. As an example, we consider a reinforcement learning application, and learn a risk-sensitive controller for the game of Tetris.

📄 PDF Abstract BibTeX arXiv:1404.3862

Code (1)

ido90/CeSoR pytorch

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Optimizing Conditional Value-At-Risk of Black-Box Functions

2021-12-01 · NeurIPS 2021 12 · Quoc Phong Nguyen, Zhongxiang Dai, Bryan Kian Hsiang Low, Patrick Jaillet

This paper presents two Bayesian optimization (BO) algorithms with theoretical performance guarantee to maximize the conditional value-at-risk (CVaR) of a black-box function: CV-UCB and CV-TS which are based on the well-…

Bayesian Optimization Thompson Sampling

Adaptive Sampling for Stochastic Risk-Averse Learning

2019-10-28 · NeurIPS 2020 12 · Sebastian Curi, Kfir. Y. Levy, Stefanie Jegelka, Andreas Krause

In high-stakes machine learning applications, it is crucial to not only perform well on average, but also when restricted to difficult examples. To address this, we consider the problem of training models in a risk-avers…

Point Processes

Risk-Sensitive Markov Decision Processes with Long-Run CVaR Criterion

2022-10-17 · Li Xia, Peter W. Glynn

CVaR (Conditional Value at Risk) is a risk metric widely used in finance. However, dynamically optimizing CVaR is difficult since it is not a standard Markov decision process (MDP) and the principle of dynamic programmin…

Management Sensitivity

Boosting CVaR Policy Optimization with Quantile Gradients

2026-01-29 · Yudong Luo, Erick Delage arxiv

Optimizing Conditional Value-at-risk (CVaR) using policy gradient (a.k.a CVaR-PG) faces significant challenges of sample inefficiency. This inefficiency stems from the fact that it focuses on tail-end performance and ove…

Robust Stochastic Shortest-Path Planning via Risk-Sensitive Incremental Sampling

2024-08-16 · Clinton Enwerem, Erfaun Noorani, John S. Baras, Brian M. Sadler

With the pervasiveness of Stochastic Shortest-Path (SSP) problems in high-risk industries, such as last-mile autonomous delivery and supply chain management, robust planning algorithms are crucial for ensuring successful…