Stochastic Optimization for Spectral Risk Measures
Spectral risk objectives - also called $L$-risks - allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop stochastic algorithms to optimize these quantities by characterizing their subdifferential and addressing challenges such as biasedness of subgradient estimates and non-smoothness of the objective. We show theoretically and experimentally that out-of-the-box approaches such as stochastic subgradient and dual averaging are hindered by bias and that our approach outperforms them.
Code (1)
Tasks
Stochastic OptimizationSimilar Papers 제목 키워드 기반
Non-asymptotic estimation of risk measures using stochastic gradient Langevin dynamics
In this paper we will study the approximation of arbitrary law invariant risk measures. As a starting point, we approximate the average value at risk using stochastic gradient Langevin dynamics, which can be seen as a va…
Higher order measures of risk and stochastic dominance
Higher order risk measures are stochastic optimization problems by design, and for this reason they enjoy valuable properties in optimization under uncertainties. They nicely integrate with stochastic optimization proble…
Stochastic OptimizationSpectral-Risk Safe Reinforcement Learning with Convergence Guarantees
The field of risk-constrained reinforcement learning (RCRL) has been developed to effectively reduce the likelihood of worst-case scenarios by explicitly handling risk-measure-based constraints. However, the nonlinearity…
Bilevel Optimizationcontinuous-controlContinuous Controlreinforcement-learning+2Safe RLHF Beyond Expectation: Stochastic Dominance for Universal Spectral Risk Control
Safe Reinforcement Learning from Human Feedback (RLHF) typically enforces safety through expected cost constraints, but the expectation captures only a single statistic of the cost distribution and fails to account for d…
Reinforcement LearningClosed-form solutions for worst-case law invariant risk measures with application to robust portfolio optimization
Worst-case risk measures refer to the calculation of the largest value for risk measures when only partial information of the underlying distribution is available. For the popular risk measures such as Value-at-Risk (VaR…
FormPortfolio Optimization