Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces
Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box functions. However, in settings with very few function evaluations, a successful application of BO may require transferring information from historical experiments. These related experiments may not have exactly the same tunable parameters (search spaces), motivating the need for BO with transfer learning for heterogeneous search spaces. In this paper, we propose two methods for this setting. The first approach leverages a Gaussian process (GP) model with a conditional kernel to transfer information between different search spaces. Our second approach treats the missing parameters as hyperparameters of the GP model that can be inferred jointly with the other GP hyperparameters or set to fixed values. We show that these two methods perform well on several benchmark problems.
Code (1)
Tasks
Bayesian OptimizationTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Transfer Learning for Bayesian Optimization on Heterogeneous Search Spaces
Bayesian optimization (BO) is a popular black-box function optimization method, which makes sequential decisions based on a Bayesian model, typically a Gaussian process (GP), of the function. To ensure the quality of the…
Bayesian OptimizationTransfer LearningTransfer Learning in Bayesian Optimization for Aircraft Design
The use of transfer learning within Bayesian optimization addresses the disadvantages of the so-called \textit{cold start} problem by using source data to aid in the optimization of a target problem. We present a method …
Transfer LearningBayesian Inverse Transfer in Evolutionary Multiobjective Optimization
Transfer optimization enables data-efficient optimization of a target task by leveraging experiential priors from related source tasks. This is especially useful in multiobjective optimization settings where a set of tra…
Bayesian OptimizationMultiobjective OptimizationTransfer LearningEfficient Bayesian Optimization with Deep Kernel Learning and Transformer Pre-trained on Multiple Heterogeneous Datasets
Bayesian optimization (BO) is widely adopted in black-box optimization problems and it relies on a surrogate model to approximate the black-box response function. With the increasing number of black-box optimization task…
Bayesian OptimizationTransfer Learning for Bayesian Optimization: A Survey
A wide spectrum of design and decision problems, including parameter tuning, A/B testing and drug design, intrinsically are instances of black-box optimization. Bayesian optimization (BO) is a powerful tool that models a…
Bayesian OptimizationDrug DesignSurveyTransfer Learning