Additive Tree-Structured Covariance Function for Conditional Parameter Spaces in Bayesian Optimization
Bayesian optimization (BO) is a sample-efficient global optimization algorithm for black-box functions which are expensive to evaluate. Existing literature on model based optimization in conditional parameter spaces are usually built on trees. In this work, we generalize the additive assumption to tree-structured functions and propose an additive tree-structured covariance function, showing improved sample-efficiency, wider applicability and greater flexibility. Furthermore, by incorporating the structure information of parameter spaces and the additive assumption in the BO loop, we develop a parallel algorithm to optimize the acquisition function and this optimization can be performed in a low dimensional space. We demonstrate our method on an optimization benchmark function, as well as on a neural network model compression problem, and experimental results show our approach significantly outperforms the current state of the art for conditional parameter optimization including SMAC, TPE and Jenatton et al. (2017).
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian Optimizationglobal-optimizationModel CompressionSMACSMAC+Similar Papers 제목 키워드 기반
Additive Tree-Structured Conditional Parameter Spaces in Bayesian Optimization: A Novel Covariance Function and a Fast Implementation
Bayesian optimization (BO) is a sample-efficient global optimization algorithm for black-box functions which are expensive to evaluate. Existing literature on model based optimization in conditional parameter spaces are …
Bayesian Optimizationglobal-optimizationNeural Network CompressionSMAC+1Robust estimation of tree structured Gaussian Graphical Model
Consider jointly Gaussian random variables whose conditional independence structure is specified by a graphical model. If we observe realizations of the variables, we can compute the covariance matrix, and it is well kno…
modelGeneralized Bayesian Additive Regression Trees Models: Beyond Conditional Conjugacy
Bayesian additive regression trees have seen increased interest in recent years due to their ability to combine machine learning techniques with principled uncertainty quantification. The Bayesian backfitting algorithm u…
regressionSurvival AnalysisUncertainty QuantificationOn the Gaussian process limit of Bayesian Additive Regression Trees
Bayesian Additive Regression Trees (BART) is a nonparametric Bayesian regression technique of rising fame. It is a sum-of-decision-trees model, and is in some sense the Bayesian version of boosting. In the limit of infin…
regressionStatistically Optimal Structured Additive MIMO Continuous-time System Identification
Many applications in mechanical, acoustic, and electronic engineering require estimating complex dynamical models, often represented as additive multi-input multi-output (MIMO) transfer functions with structural constrai…