Approximation-Aware Bayesian Optimization
High-dimensional Bayesian optimization (BO) tasks such as molecular design often require 10,000 function evaluations before obtaining meaningful results. While methods like sparse variational Gaussian processes (SVGPs) reduce computational requirements in these settings, the underlying approximations result in suboptimal data acquisitions that slow the progress of optimization. In this paper we modify SVGPs to better align with the goals of BO: targeting informed data acquisition rather than global posterior fidelity. Using the framework of utility-calibrated variational inference, we unify GP approximation and data acquisition into a joint optimization problem, thereby ensuring optimal decisions under a limited computational budget. Our approach can be used with any decision-theoretic acquisition function and is compatible with trust region methods like TuRBO. We derive efficient joint objectives for the expected improvement and knowledge gradient acquisition functions in both the standard and batch BO settings. Our approach outperforms standard SVGPs on high-dimensional benchmark tasks in control and molecular design.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationGaussian ProcessesVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fair and Green Hyperparameter Optimization via Multi-objective and Multiple Information Source Bayesian Optimization
There is a consensus that focusing only on accuracy in searching for optimal machine learning models amplifies biases contained in the data, leading to unfair predictions and decision supports. Recently, multi-objective …
Bayesian OptimizationBIG-bench Machine LearningFairnessHyperparameter OptimizationScalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes
Bayesian optimization is a technique for optimizing black-box target functions. At the core of Bayesian optimization is a surrogate model that predicts the output of the target function at previously unseen inputs to fac…
Bayesian OptimizationGaussian ProcessesThompson SamplingGuided Multi-Fidelity Bayesian Optimization for Data-driven Controller Tuning with Digital Twins
We propose a \textit{guided multi-fidelity Bayesian optimization} framework for data-efficient controller tuning that integrates corrected digital twin simulations with real-world measurements. The method targets closed-…
When can we improve on sample average approximation for stochastic optimization?
We explore the performance of sample average approximation in comparison with several other methods for stochastic optimization when there is information available on the underlying true probability distribution. The met…
Portfolio OptimizationStochastic OptimizationResource Aware Multifidelity Active Learning for Efficient Optimization
Traditional methods for black box optimization require a considerable number of evaluations which can be time consuming, unpractical, and often unfeasible for many engineering applications that rely on accurate represent…
Active LearningBayesian OptimizationDistributed Computing