Bayesian Optimization using Pseudo-Points
Bayesian optimization (BO) is a popular approach for expensive black-box optimization, with applications including parameter tuning, experimental design, robotics. BO usually models the objective function by a Gaussian process (GP), and iteratively samples the next data point by maximizing an acquisition function. In this paper, we propose a new general framework for BO by generating pseudo-points (i.e., data points whose objective values are not evaluated) to improve the GP model. With the classic acquisition function, i.e., upper confidence bound (UCB), we prove that the cumulative regret can be generally upper bounded. Experiments using UCB and other acquisition functions, i.e., probability of improvement (PI) and expectation of improvement (EI), on synthetic as well as real-world problems clearly show the advantage of generating pseudo-points.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationExperimental DesignMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bayesian Pseudocoresets
Standard Bayesian inference algorithms are prohibitively expensive in the regime of modern large-scale data. Recent work has found that a small, weighted subset of data (a coreset) may be used in place of the full datase…
Bayesian InferencePseudo-Bayesian Optimization
Bayesian Optimization is a popular approach for optimizing expensive black-box functions. Its key idea is to use a surrogate model to approximate the objective and, importantly, quantify the associated uncertainty that a…
Bayesian OptimizationUncertainty QuantificationSparse Gaussian processes using pseudo-inputs
We present a new Gaussian process (GP) regression model whose covariance is parameterized by the the locations of M pseudo-input points, which we learn by a gradient based optimization. We take M << N, where N is the num…
Gaussian ProcessesregressionOn Divergence Measures for Bayesian Pseudocoresets
A Bayesian pseudocoreset is a small synthetic dataset for which the posterior over parameters approximates that of the original dataset. While promising, the scalability of Bayesian pseudocoresets is not yet validated in…
Bayesian InferenceDataset Distillationimage-classificationImage ClassificationBayesian Optimization with a Prior for the Optimum
While Bayesian Optimization (BO) is a very popular method for optimizing expensive black-box functions, it fails to leverage the experience of domain experts. This causes BO to waste function evaluations on bad design ch…
Bayesian Optimization