Random Exploration in Bayesian Optimization: Order-Optimal Regret and Computational Efficiency
We consider Bayesian optimization using Gaussian Process models, also referred to as kernel-based bandit optimization. We study the methodology of exploring the domain using random samples drawn from a distribution. We show that this random exploration approach achieves the optimal error rates. Our analysis is based on novel concentration bounds in an infinite dimensional Hilbert space established in this work, which may be of independent interest. We further develop an algorithm based on random exploration with domain shrinking and establish its order-optimal regret guarantees under both noise-free and noisy settings. In the noise-free setting, our analysis closes the existing gap in regret performance and thereby resolves a COLT open problem. The proposed algorithm also enjoys a computational advantage over prevailing methods due to the random exploration that obviates the expensive optimization of a non-convex acquisition function for choosing the query points at each iteration.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationComputational EfficiencyMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enhancing Gaussian Process Surrogates for Optimization and Posterior Approximation via Random Exploration
This paper proposes novel noise-free Bayesian optimization strategies that rely on a random exploration step to enhance the accuracy of Gaussian process surrogate models. The new algorithms retain the ease of implementat…
Bayesian InferenceBayesian OptimizationBayesian Optimization for Unknown Cost-Varying Variable Subsets with No-Regret Costs
Bayesian Optimization (BO) is a widely-used method for optimizing expensive-to-evaluate black-box functions. Traditional BO assumes that the learner has full control over all query variables without additional constraint…
Bayesian OptimizationTopological Bayesian Optimization with Persistence Diagrams
Finding an optimal parameter of a black-box function is important for searching stable material structures and finding optimal neural network structures, and Bayesian optimization algorithms are widely used for the purpo…
Bayesian OptimizationTopological Data AnalysisHyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian optimization to adaptively select configurations, we focus on speeding up r…
Bayesian OptimizationHyperparameter OptimizationPreparation of ordered states in ultra-cold gases using Bayesian optimization
Ultra-cold atomic gases are unique in terms of the degree of controllability, both for internal and external degrees of freedom. This makes it possible to use them for the study of complex quantum many-body phenomena. Ho…
Bayesian Optimization