High Dimensional Bayesian Optimization via Supervised Dimension Reduction
Bayesian optimization (BO) has been broadly applied to computational expensive problems, but it is still challenging to extend BO to high dimensions. Existing works are usually under strict assumption of an additive or a linear embedding structure for objective functions. This paper directly introduces a supervised dimension reduction method, Sliced Inverse Regression (SIR), to high dimensional Bayesian optimization, which could effectively learn the intrinsic sub-structure of objective function during the optimization. Furthermore, a kernel trick is developed to reduce computational complexity and learn nonlinear subset of the unknowing function when applying SIR to extremely high dimensional BO. We present several computational benefits and derive theoretical regret bounds of our algorithm. Extensive experiments on synthetic examples and two real applications demonstrate the superiority of our algorithms for high dimensional Bayesian optimization.
Code (1)
Tasks
Bayesian OptimizationDimensionality ReductionVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Semi-supervised Embedding Learning for High-dimensional Bayesian Optimization
Bayesian optimization is a broadly applied methodology to optimize the expensive black-box function. Despite its success, it still faces the challenge from the high-dimensional search space. To alleviate this problem, we…
Bayesian OptimizationDimensionality ReductionHyperparameter OptimizationVocal Bursts Intensity PredictionGood practices for Bayesian Optimization of high dimensional structured spaces
The increasing availability of structured but high dimensional data has opened new opportunities for optimization. One emerging and promising avenue is the exploration of unsupervised methods for projecting structured hi…
Bayesian OptimizationVocal Bursts Intensity PredictionEfficient and Scalable Batch Bayesian Optimization Using K-Means
We present K-Means Batch Bayesian Optimization (KMBBO), a novel batch sampling algorithm for Bayesian Optimization (BO). KMBBO uses unsupervised learning to efficiently estimate peaks of the model acquisition function. W…
Bayesian Optimizationcompressed sensingDimensionality ReductionDrug DiscoveryHigh-Dimensional Discrete Bayesian Optimization with Self-Supervised Representation Learning for Data-Efficient Materials Exploration
A material exploration model based on high-dimensional discrete Bayesian optimization is introduced. Features were extracted from a large-scale database of ab-initio calculations by self-supervised representation learnin…
Bayesian OptimizationRepresentation LearningAn Adaptive Dropout Approach for High-Dimensional Bayesian Optimization
Bayesian optimization (BO) is a widely used algorithm for solving expensive black-box optimization problems. However, its performance decreases significantly on high-dimensional problems due to the inherent high-dimensio…
Bayesian Optimization