Global Optimization of Gaussian Process Acquisition Functions Using a Piecewise-Linear Kernel Approximation
Bayesian optimization relies on iteratively constructing and optimizing an acquisition function. The latter turns out to be a challenging, non-convex optimization problem itself. Despite the relative importance of this step, most algorithms employ sampling- or gradient-based methods, which do not provably converge to global optima. This work investigates mixed-integer programming (MIP) as a paradigm for \textit{global} acquisition function optimization. Specifically, our Piecewise-linear Kernel Mixed Integer Quadratic Programming (PK-MIQP) formulation introduces a piecewise-linear approximation for Gaussian process kernels and admits a corresponding MIQP representation for acquisition functions. We analyze the theoretical regret bounds of the proposed approximation, and empirically demonstrate the framework on synthetic functions, constrained benchmarks, and a hyperparameter tuning task.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian Optimizationglobal-optimizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Global Optimization of Gaussian processes
Gaussian processes~(Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in…
Bayesian OptimizationGaussian Processesglobal-optimizationBayesian Multi-Scale Optimistic Optimization
Bayesian optimization is a powerful global optimization technique for expensive black-box functions. One of its shortcomings is that it requires auxiliary optimization of an acquisition function at each iteration. This a…
Bayesian OptimizationGaussian Processesglobal-optimizationOn Local Optimizers of Acquisition Functions in Bayesian Optimization
Bayesian optimization is a sample-efficient method for finding a global optimum of an expensive-to-evaluate black-box function. A global solution is found by accumulating a pair of query point and its function value, rep…
Bayesian OptimizationvalidThe reparameterization trick for acquisition functions
Bayesian optimization is a sample-efficient approach to solving global optimization problems. Along with a surrogate model, this approach relies on theoretically motivated value heuristics (acquisition functions) to guid…
Bayesian Optimizationglobal-optimizationBatched Energy-Entropy acquisition for Bayesian Optimization
Bayesian optimization (BO) is an attractive machine learning framework for performing sample-efficient global optimization of black-box functions. The optimization process is guided by an acquisition function that select…
Bayesian OptimizationGaussian Processesglobal-optimization