Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection
Bayesian Optimization (BO) is a method for globally optimizing black-box functions. While BO has been successfully applied to many scenarios, developing effective BO algorithms that scale to functions with high-dimensional domains is still a challenge. Optimizing such functions by vanilla BO is extremely time-consuming. Alternative strategies for high-dimensional BO that are based on the idea of embedding the high-dimensional space to the one with low dimension are sensitive to the choice of the embedding dimension, which needs to be pre-specified. We develop a new computationally efficient high-dimensional BO method that exploits variable selection. Our method is able to automatically learn axis-aligned sub-spaces, i.e. spaces containing selected variables, without the demand of any pre-specified hyperparameters. We theoretically analyze the computational complexity of our algorithm and derive the regret bound. We empirically show the efficacy of our method on several synthetic and real problems.
Code (1)
Tasks
Bayesian OptimizationVariable SelectionVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm
Bayesian variable selection methods are powerful techniques for fitting and inferring on sparse high-dimensional linear regression models. However, many are computationally intensive or require restrictive prior distribu…
Prediction IntervalsregressionVariable SelectionProbabilistic Bayesian optimal experimental design using conditional normalizing flows
Bayesian optimal experimental design (OED) seeks to conduct the most informative experiment under budget constraints to update the prior knowledge of a system to its posterior from the experimental data in a Bayesian fra…
Experimental DesignFine-tuning LLMs with variational Bayesian last layer for high-dimensional Bayesian optimization
A plethora of applications entail solving black-box optimization problems with high evaluation costs, including drug discovery, material design, as well as hyperparameter tuning. Toward finding the global optimum of such…
Drug DiscoveryStructured Bayesian Gaussian process latent variable model: applications to data-driven dimensionality reduction and high-dimensional inversion
We introduce a methodology for nonlinear inverse problems using a variational Bayesian approach where the unknown quantity is a spatial field. A structured Bayesian Gaussian process latent variable model is used both to …
Dimensionality ReductionHigh Dimensional Bayesian Optimization Using Dropout
Scaling Bayesian optimization to high dimensions is challenging task as the global optimization of high-dimensional acquisition function can be expensive and often infeasible. Existing methods depend either on limited ac…
Bayesian Optimizationglobal-optimizationVocal Bursts Intensity Prediction