On Optimizing the Conditional Value-at-Risk of a Maximum Cost for Risk-Averse Safety Analysis
The popularity of Conditional Value-at-Risk (CVaR), a risk functional from finance, has been growing in the control systems community due to its intuitive interpretation and axiomatic foundation. We consider a nonstandard optimal control problem in which the goal is to minimize the CVaR of a maximum random cost subject to a Borel-space Markov decision process. The objective represents the maximum departure from a desired operating region averaged over a given fraction of the worst cases. This problem provides a safety criterion for a stochastic system that is informed by both the probability and severity of the potential consequences of the system's behavior. In contrast, existing safety analysis frameworks apply stage-wise risk constraints or assess the probability of constraint violation without quantifying the potential severity of the violation. To the best of our knowledge, the problem of interest has not been solved. To solve the problem, we propose and study a family of stochastic dynamic programs on an augmented state space. We prove that the optimal CVaR of a maximum random cost enjoys an equivalent representation in terms of the solutions to these dynamic programs under appropriate assumptions. For each dynamic program, we show the existence of an optimal policy that depends on the dynamics of an augmented state under the assumptions. In a numerical example, we illustrate how our safety analysis framework is useful for assessing the severity of combined sewer overflows under precipitation uncertainty.
Code (1)
Similar Papers 제목 키워드 기반
Risk-sensitive safety analysis using Conditional Value-at-Risk
This paper develops a safety analysis method for stochastic systems that is sensitive to the possibility and severity of rare harmful outcomes. We define risk-sensitive safe sets as sub-level sets of the solution to a no…
Diversified reward-risk parity in portfolio construction
We introduce diversified risk parity embedded with various reward-risk measures and more generic allocation rules for portfolio construction. We empirically test the proposed reward-risk parity strategies and compare the…
A Noise-Robust Elicit-to-Optimize Framework for Distortion Riskmetrics via Inverse Reinforcement Learning
We propose a noise-robust elicit-to-optimize framework that integrates inverse reinforcement learning (IRL) and reinforcement learning (RL) for eliciting agents' risk preferences and optimizing policies under a broad cla…
Reinforcement LearningOptimizing Conditional Value-At-Risk of Black-Box Functions
This paper presents two Bayesian optimization (BO) algorithms with theoretical performance guarantee to maximize the conditional value-at-risk (CVaR) of a black-box function: CV-UCB and CV-TS which are based on the well-…
Bayesian OptimizationThompson SamplingRisk-Sensitive Reinforcement Learning via Policy Gradient Search
The objective in a traditional reinforcement learning (RL) problem is to find a policy that optimizes the expected value of a performance metric such as the infinite-horizon cumulative discounted or long-run average cost…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)