Transfer Learning for Bayesian Optimization on Heterogeneous Search Spaces
Bayesian optimization (BO) is a popular black-box function optimization method, which makes sequential decisions based on a Bayesian model, typically a Gaussian process (GP), of the function. To ensure the quality of the model, transfer learning approaches have been developed to automatically design GP priors by learning from observations on "training" functions. These training functions are typically required to have the same domain as the "test" function (black-box function to be optimized). In this paper, we introduce MPHD, a model pre-training method on heterogeneous domains, which uses a neural net mapping from domain-specific contexts to specifications of hierarchical GPs. MPHD can be seamlessly integrated with BO to transfer knowledge across heterogeneous search spaces. Our theoretical and empirical results demonstrate the validity of MPHD and its superior performance on challenging black-box function optimization tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces
Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box functions. However, in settings with very few function evaluations, a successful application of BO may require transferring …
Bayesian OptimizationTransfer LearningBayesian Inverse Transfer in Evolutionary Multiobjective Optimization
Transfer optimization enables data-efficient optimization of a target task by leveraging experiential priors from related source tasks. This is especially useful in multiobjective optimization settings where a set of tra…
Bayesian OptimizationMultiobjective OptimizationTransfer LearningEfficient Bayesian Optimization with Deep Kernel Learning and Transformer Pre-trained on Multiple Heterogeneous Datasets
Bayesian optimization (BO) is widely adopted in black-box optimization problems and it relies on a surrogate model to approximate the black-box response function. With the increasing number of black-box optimization task…
Bayesian OptimizationHyperBO+: Pre-training a universal prior for Bayesian optimization with hierarchical Gaussian processes
Bayesian optimization (BO), while proved highly effective for many black-box function optimization tasks, requires practitioners to carefully select priors that well model their functions of interest. Rather than specify…
Bayesian OptimizationGaussian ProcessesTransfer LearningLearning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
Bayesian optimization (BO) is a successful methodology to optimize black-box functions that are expensive to evaluate. While traditional methods optimize each black-box function in isolation, there has been recent intere…
Bayesian OptimizationHyperparameter OptimizationTransfer Learning