On Exponential Utility and Conditional Value-at-Risk as Risk-Averse Performance Criteria
The standard approach to risk-averse control is to use the Exponential Utility (EU) functional, which has been studied for several decades. Like other risk-averse utility functionals, EU encodes risk aversion through an increasing convex mapping $\varphi$ of objective costs to subjective costs. An objective cost is a realization $y$ of a random variable $Y$. In contrast, a subjective cost is a realization $\varphi(y)$ of a random variable $\varphi(Y)$ that has been transformed to measure preferences about the outcomes. For EU, the transformation is $\varphi(y) = \exp(\frac{-\theta}{2}y)$, and under certain conditions, the quantity $\varphi^{-1}(E(\varphi(Y)))$ can be approximated by a linear combination of the mean and variance of $Y$. More recently, there has been growing interest in risk-averse control using the Conditional Value-at-Risk (CVaR) functional. In contrast to the EU functional, the CVaR of a random variable $Y$ concerns a fraction of its possible realizations. If $Y$ is a continuous random variable with finite $E(|Y|)$, then the CVaR of $Y$ at level $\alpha$ is the expectation of $Y$ in the $\alpha \cdot 100 \%$ worst cases. Here, we study the applications of risk-averse functionals to controller synthesis and safety analysis through the development of numerical examples, with emphasis on EU and CVaR. Our contribution is to examine the decision-theoretic, mathematical, and computational trade-offs that arise when using EU and CVaR for optimal control and safety analysis. We are hopeful that this work will advance the interpretability and elucidate the potential benefits of risk-averse control technology.
Code (1)
Similar Papers 제목 키워드 기반
Risk-Sensitive Reinforcement Learning via Policy Gradient Search
The objective in a traditional reinforcement learning (RL) problem is to find a policy that optimizes the expected value of a performance metric such as the infinite-horizon cumulative discounted or long-run average cost…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)A Wasserstein distance approach for concentration of empirical risk estimates
This paper presents a unified approach based on Wasserstein distance to derive concentration bounds for empirical estimates for two broad classes of risk measures defined in the paper. The classes of risk measures introd…
Expected Utility Maximization and Conditional Value-at-Risk Deviation-based Sharpe Ratio in Dynamic Stochastic Portfolio Optimization
In this paper we investigate the expected terminal utility maximization approach for a dynamic stochastic portfolio optimization problem. We solve it numerically by solving an evolutionary Hamilton-Jacobi-Bellman equatio…
Portfolio OptimizationMean-field equilibrium price formation with exponential utility
In this paper, using the mean-field game theory, we study a problem of equilibrium price formation among many investors with exponential utility in the presence of liabilities unspanned by the security prices. The invest…
Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret
We study risk-sensitive reinforcement learning in episodic Markov decision processes with unknown transition kernels, where the goal is to optimize the total reward under the risk measure of exponential utility. We propo…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)