Bayesian Optimization with Output-Weighted Optimal Sampling
In Bayesian optimization, accounting for the importance of the output relative to the input is a crucial yet challenging exercise, as it can considerably improve the final result but often involves inaccurate and cumbersome entropy estimations. We approach the problem from the perspective of importance-sampling theory, and advocate the use of the likelihood ratio to guide the search algorithm towards regions of the input space where the objective function to be minimized assumes abnormally small values. The likelihood ratio acts as a sampling weight and can be computed at each iteration without severely deteriorating the overall efficiency of the algorithm. In particular, it can be approximated in a way that makes the approach tractable in high dimensions. The "likelihood-weighted" acquisition functions introduced in this work are found to outperform their unweighted counterparts in a number of applications.
Code (1)
Tasks
Bayesian OptimizationSimilar Papers 제목 키워드 기반
Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samples
For many important problems the quantity of interest is an unknown function of the parameters, which is a random vector with known statistics. Since the dependence of the output on this random vector is unknown, the chal…
Active LearningExperimental DesignregressionAdaptive posterior distributions for uncertainty analysis of covariance matrices in Bayesian inversion problems for multioutput signals
In this paper we address the problem of performing Bayesian inference for the parameters of a nonlinear multi-output model and the covariance matrix of the different output signals. We propose an adaptive importance samp…
Bayesian InferenceBayesian Optimization of Composite Functions
We consider optimization of composite objective functions, i.e., of the form $f(x)=g(h(x))$, where $h$ is a black-box derivative-free expensive-to-evaluate function with vector-valued outputs, and $g$ is a cheap-to-evalu…
Bayesian OptimizationGenerative Bayesian Hyperparameter Tuning
\noindent Hyper-parameter selection is a central practical problem in modern machine learning, governing regularization strength, model capacity, and robustness choices. Cross-validation is often computationally prohibit…
Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantification
We introduce a class of acquisition functions for sample selection that leads to faster convergence in applications related to Bayesian experimental design and uncertainty quantification. The approach follows the paradig…
Active LearningExperimental DesignUncertainty Quantification