Good practices for Bayesian Optimization of high dimensional structured spaces
The increasing availability of structured but high dimensional data has opened new opportunities for optimization. One emerging and promising avenue is the exploration of unsupervised methods for projecting structured high dimensional data into low dimensional continuous representations, simplifying the optimization problem and enabling the application of traditional optimization methods. However, this line of research has been purely methodological with little connection to the needs of practitioners so far. In this paper, we study the effect of different search space design choices for performing Bayesian Optimization in high dimensional structured datasets. In particular, we analyse the influence of the dimensionality of the latent space, the role of the acquisition function and evaluate new methods to automatically define the optimization bounds in the latent space. Finally, based on experimental results using synthetic and real datasets, we provide recommendations for the practitioners.
Code (1)
Tasks
Bayesian OptimizationVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Expected Coordinate Improvement for High-Dimensional Bayesian Optimization
Bayesian optimization (BO) algorithm is very popular for solving low-dimensional expensive optimization problems. Extending Bayesian optimization to high dimension is a meaningful but challenging task. One of the major c…
Bayesian OptimizationRevisiting Bayesian Optimization in the light of the COCO benchmark
It is commonly believed that Bayesian optimization (BO) algorithms are highly efficient for optimizing numerically costly functions. However, BO is not often compared to widely different alternatives, and is mostly teste…
Bayesian OptimizationThe Behavior and Convergence of Local Bayesian Optimization
A recent development in Bayesian optimization is the use of local optimization strategies, which can deliver strong empirical performance on high-dimensional problems compared to traditional global strategies. The "folk …
Bayesian OptimizationOPBO: Order-Preserving Bayesian Optimization
Bayesian optimization is an effective method for solving expensive black-box optimization problems. Most existing methods use Gaussian processes (GP) as the surrogate model for approximating the black-box objective funct…
Gaussian ProcessesLearning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
High dimensional black-box optimization has broad applications but remains a challenging problem to solve. Given a set of samples $\{\vx_i, y_i\}$, building a global model (like Bayesian Optimization (BO)) suffers from t…
Bayesian OptimizationNeural Architecture Search