Practical Transfer Learning for Bayesian Optimization
When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemble model for Bayesian optimization that is a generalization of two existing transfer learning extensions to Bayesian optimization and establish a worst-case bound compared to vanilla Bayesian optimization. Using a large collection of hyperparameter optimization benchmark problems, we demonstrate that our contributions substantially reduce optimization time compared to standard Gaussian process-based Bayesian optimization and improve over the current state-of-the-art for transfer hyperparameter optimization.
Code (2)
Tasks
Bayesian OptimizationGaussian ProcessesHyperparameter OptimizationMeta-LearningTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bayesian Inverse Transfer in Evolutionary Multiobjective Optimization
Transfer optimization enables data-efficient optimization of a target task by leveraging experiential priors from related source tasks. This is especially useful in multiobjective optimization settings where a set of tra…
Bayesian OptimizationMultiobjective OptimizationTransfer LearningBayMOTH: Bayesian optiMizatiOn with meTa-lookahead -- a simple approacH
Bayesian optimization (BO) has for sequential optimization of expensive black-box functions demonstrated practicality and effectiveness in many real-world settings. Meta-Bayesian optimization (meta-BO) focuses on improvi…
Transfer Learning in Bayesian Optimization for Aircraft Design
The use of transfer learning within Bayesian optimization addresses the disadvantages of the so-called \textit{cold start} problem by using source data to aid in the optimization of a target problem. We present a method …
Transfer LearningTransfer Learning for Bayesian Optimization: A Survey
A wide spectrum of design and decision problems, including parameter tuning, A/B testing and drug design, intrinsically are instances of black-box optimization. Bayesian optimization (BO) is a powerful tool that models a…
Bayesian OptimizationDrug DesignSurveyTransfer LearningRover: An online Spark SQL tuning service via generalized transfer learning
Distributed data analytic engines like Spark are common choices to process massive data in industry. However, the performance of Spark SQL highly depends on the choice of configurations, where the optimal ones vary with …
Bayesian OptimizationTransfer Learning