Quantile Learn-Then-Test: Quantile-Based Risk Control for Hyperparameter Optimization
The increasing adoption of Artificial Intelligence (AI) in engineering problems calls for the development of calibration methods capable of offering robust statistical reliability guarantees. The calibration of black box AI models is carried out via the optimization of hyperparameters dictating architecture, optimization, and/or inference configuration. Prior work has introduced learn-then-test (LTT), a calibration procedure for hyperparameter optimization (HPO) that provides statistical guarantees on average performance measures. Recognizing the importance of controlling risk-aware objectives in engineering contexts, this work introduces a variant of LTT that is designed to provide statistical guarantees on quantiles of a risk measure. We illustrate the practical advantages of this approach by applying the proposed algorithm to a radio access scheduling problem.
Code (0)
등록된 구현이 없습니다.
Tasks
Hyperparameter OptimizationSchedulingSimilar Papers 제목 키워드 기반
Quantile Geometry Regularization for Distributional Reinforcement Learning
Quantile-based distributional reinforcement learning methods learn return distributions through sampled quantile regression, but their bootstrapped target quantiles may induce distorted or degenerate distribution estimat…
Reinforcement LearningAtari GamesCommon Idiosyncratic Quantile Risk
We identify a new type of risk that is characterised by commonalities in the quantiles of the cross-sectional distribution of asset returns. Our newly proposed quantile risk factors are associated with a quantile-specifi…
Quantiles under ambiguity and risk sharing
Choquet capacities and integrals are central concepts in decision making under ambiguity or model uncertainty, pioneered by Schmeidler. Motivated by risk optimization problems for quantiles under ambiguity, we study the …
Decision MakingRisk level dependent Minimax Quantile lower bounds for Interactive Statistical Decision Making
Minimax risk and regret focus on expectation, missing rare failures critical in safety-critical bandits and reinforcement learning. Minimax quantiles capture these tails. Three strands of prior work motivate this study: …
Reinforcement LearningDecision MakingRobust Bond Risk Premia Predictability Test in the Quantiles
Different from existing literature on testing the macro-spanning hypothesis of bond risk premia, which only considers mean regressions, this paper investigates whether the yield curve represented by CP factor (Cochrane a…
quantile regression