TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning
With the extensive applications of machine learning models, automatic hyperparameter optimization (HPO) has become increasingly important. Motivated by the tuning behaviors of human experts, it is intuitive to leverage auxiliary knowledge from past HPO tasks to accelerate the current HPO task. In this paper, we propose TransBO, a novel two-phase transfer learning framework for HPO, which can deal with the complementary nature among source tasks and dynamics during knowledge aggregation issues simultaneously. This framework extracts and aggregates source and target knowledge jointly and adaptively, where the weights can be learned in a principled manner. The extensive experiments, including static and dynamic transfer learning settings and neural architecture search, demonstrate the superiority of TransBO over the state-of-the-arts.
Code (0)
등록된 구현이 없습니다.
Tasks
Hyperparameter OptimizationNeural Architecture SearchTransfer LearningVocal Bursts Valence PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
TransBoost: A Boosting-Tree Kernel Transfer Learning Algorithm for Improving Financial Inclusion
The prosperity of mobile and financial technologies has bred and expanded various kinds of financial products to a broader scope of people, which contributes to advocating financial inclusion. It has non-trivial social b…
Transfer LearningTransBoost: Improving the Best ImageNet Performance using Deep Transduction
This paper deals with deep transductive learning, and proposes TransBoost as a procedure for fine-tuning any deep neural model to improve its performance on any (unlabeled) test set provided at training time. TransBoost …
image-classificationImage ClassificationTransductive LearningImproving Multi-fidelity Optimization with a Recurring Learning Rate for Hyperparameter Tuning
Despite the evolution of Convolutional Neural Networks (CNNs), their performance is surprisingly dependent on the choice of hyperparameters. However, it remains challenging to efficiently explore large hyperparameter sea…
image-classificationImage ClassificationTransfer LearningWarm Starting CMA-ES for Hyperparameter Optimization
Hyperparameter optimization (HPO), formulated as black-box optimization (BBO), is recognized as essential for automation and high performance of machine learning approaches. The CMA-ES is a promising BBO approach with a …
Bayesian OptimizationHyperparameter OptimizationTransfer LearningPractical Transfer Learning for Bayesian Optimization
When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemb…
Bayesian OptimizationGaussian ProcessesHyperparameter OptimizationMeta-Learning+1