High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups
Bayesian optimization (BO) is a popular technique for sequential black-box function optimization, with applications including parameter tuning, robotics, environmental monitoring, and more. One of the most important challenges in BO is the development of algorithms that scale to high dimensions, which remains a key open problem despite recent progress. In this paper, we consider the approach of Kandasamy et al. (2015), in which the high-dimensional function decomposes as a sum of lower-dimensional functions on subsets of the underlying variables. In particular, we significantly generalize this approach by lifting the assumption that the subsets are disjoint, and consider additive models with arbitrary overlap among the subsets. By representing the dependencies via a graph, we deduce an efficient message passing algorithm for optimizing the acquisition function. In addition, we provide an algorithm for learning the graph from samples based on Gibbs sampling. We empirically demonstrate the effectiveness of our methods on both synthetic and real-world data.
Code (1)
Tasks
Additive modelsBayesian OptimizationHyperparameter OptimizationVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
High-Dimensional Bayesian Optimization via Tree-Structured Additive Models
Bayesian Optimization (BO) has shown significant success in tackling expensive low-dimensional black-box optimization problems. Many optimization problems of interest are high-dimensional, and scaling BO to such settings…
Additive modelsBayesian OptimizationGraph LearningVocal Bursts Intensity PredictionEfficient High Dimensional Bayesian Optimization with Additivity and Quadrature Fourier Features
We develop an efficient and provably no-regret Bayesian optimization (BO) algorithm for optimization of black-box functions in high dimensions. We assume a generalized additive model with possibly overlapping variable gr…
Bayesian OptimizationHyperparameter OptimizationNumerical IntegrationVocal Bursts Intensity PredictionRelaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimization
Bayesian Optimization (BO) is typically used to optimize an unknown function $f$ that is noisy and costly to evaluate, by exploiting an acquisition function that must be maximized at each optimization step. Even if prova…
Bayesian OptimizationHigh Dimensional Bayesian Optimization Using Dropout
Scaling Bayesian optimization to high dimensions is challenging task as the global optimization of high-dimensional acquisition function can be expensive and often infeasible. Existing methods depend either on limited ac…
Bayesian Optimizationglobal-optimizationVocal Bursts Intensity PredictionHigh Dimensional Bayesian Optimization via Supervised Dimension Reduction
Bayesian optimization (BO) has been broadly applied to computational expensive problems, but it is still challenging to extend BO to high dimensions. Existing works are usually under strict assumption of an additive or a…
Bayesian OptimizationDimensionality ReductionVocal Bursts Intensity Prediction