Risk-averse Heteroscedastic Bayesian Optimization
Many black-box optimization tasks arising in high-stakes applications require risk-averse decisions. The standard Bayesian optimization (BO) paradigm, however, optimizes the expected value only. We generalize BO to trade mean and input-dependent variance of the objective, both of which we assume to be unknown a priori. In particular, we propose a novel risk-averse heteroscedastic Bayesian optimization algorithm (RAHBO) that aims to identify a solution with high return and low noise variance, while learning the noise distribution on the fly. To this end, we model both expectation and variance as (unknown) RKHS functions, and propose a novel risk-aware acquisition function. We bound the regret for our approach and provide a robust rule to report the final decision point for applications where only a single solution must be identified. We demonstrate the effectiveness of RAHBO on synthetic benchmark functions and hyperparameter tuning tasks.
Code (1)
Tasks
Bayesian OptimizationSimilar Papers 제목 키워드 기반
Efficient Heteroscedastic Bayesian Optimization for Risk-Aware AutoRL
Reinforcement learning (RL) has shown remarkable success across a wide range of complex tasks. However, RL outcomes can be highly stochastic, and both expected performance and variability often depend on hyperparameter (…
Reinforcement LearningSafe Risk-averse Bayesian Optimization for Controller Tuning
Controller tuning and parameter optimization are crucial in system design to improve both the controller and underlying system performance. Bayesian optimization has been established as an efficient model-free method for…
Bayesian OptimizationBayesian Quantile and Expectile Optimisation
Bayesian optimisation (BO) is widely used to optimise stochastic black box functions. While most BO approaches focus on optimising conditional expectations, many applications require risk-averse strategies and alternativ…
Bayesian OptimisationGaussian ProcessesThompson SamplingBatch Bayesian Optimization for Replicable Experimental Design
Many real-world experimental design problems (a) evaluate multiple experimental conditions in parallel and (b) replicate each condition multiple times due to large and heteroscedastic observation noise. Given a fixed tot…
AutoMLBayesian OptimizationExperimental DesignThompson SamplingBayesian Risk-Averse Q-Learning with Streaming Observations
We consider a robust reinforcement learning problem, where a learning agent learns from a simulated training environment. To account for the model mis-specification between this training environment and the real environm…
Q-Learning