Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes
Bayesian optimization is a technique for optimizing black-box target functions. At the core of Bayesian optimization is a surrogate model that predicts the output of the target function at previously unseen inputs to facilitate the selection of promising input values. Gaussian processes (GPs) are commonly used as surrogate models but are known to scale poorly with the number of observations. We adapt the Vecchia approximation, a popular GP approximation from spatial statistics, to enable scalable high-dimensional Bayesian optimization. We develop several improvements and extensions, including training warped GPs using mini-batch gradient descent, approximate neighbor search, and selecting multiple input values in parallel. We focus on the use of our warped Vecchia GP in trust-region Bayesian optimization via Thompson sampling. On several test functions and on two reinforcement-learning problems, our methods compared favorably to the state of the art.
Code (1)
Tasks
Bayesian OptimizationGaussian ProcessesThompson SamplingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Scalable Gaussian-process regression and variable selection using Vecchia approximations
Gaussian process (GP) regression is a flexible, nonparametric approach to regression that naturally quantifies uncertainty. In many applications, the number of responses and covariates are both large, and a goal is to se…
regressionVariable SelectionVecchia-Inducing-Points Full-Scale Approximations for Gaussian Processes
Gaussian processes are flexible, probabilistic, non-parametric models widely used in machine learning and statistics. However, their scalability to large data sets is limited by computational constraints. To overcome the…
Gaussian ProcessesIterative Methods for Vecchia-Laplace Approximations for Latent Gaussian Process Models
Latent Gaussian process (GP) models are flexible probabilistic non-parametric function models. Vecchia approximations are accurate approximations for GPs to overcome computational bottlenecks for large data, and the Lapl…
Permutation-preserving Functions and Neural Vecchia Covariance Kernels
We introduce a novel framework for constructing scalable and flexible covariance kernels for Gaussian processes (GPs) by directly learning the covariance structure under a regression-type parameterization induced by Vecc…
Gaussian ProcessesAn accuracy-runtime trade-off comparison of scalable Gaussian process approximations for spatial data
Gaussian processes (GPs) are flexible, probabilistic, non-parametric models widely employed in various fields such as spatial statistics and machine learning. A drawback of Gaussian processes is their computational cost …
Gaussian Processesparameter estimationTime Series Analysis