paper-with-me

홈 › Papers

TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning

2022-06-06 · Yang Li, Yu Shen, Huaijun Jiang, Wentao Zhang, Zhi Yang, Ce Zhang, Bin Cui

With the extensive applications of machine learning models, automatic hyperparameter optimization (HPO) has become increasingly important. Motivated by the tuning behaviors of human experts, it is intuitive to leverage auxiliary knowledge from past HPO tasks to accelerate the current HPO task. In this paper, we propose TransBO, a novel two-phase transfer learning framework for HPO, which can deal with the complementary nature among source tasks and dynamics during knowledge aggregation issues simultaneously. This framework extracts and aggregates source and target knowledge jointly and adaptively, where the weights can be learned in a principled manner. The extensive experiments, including static and dynamic transfer learning settings and neural architecture search, demonstrate the superiority of TransBO over the state-of-the-arts.

📄 PDF Abstract BibTeX arXiv:2206.02663

Code (0)

등록된 구현이 없습니다.

Tasks

Hyperparameter OptimizationNeural Architecture SearchTransfer LearningVocal Bursts Valence Prediction

Methods 이 논문이 사용한 방법론

HPO In machine learning, a hyperparameter is a parameter whose value is used to control learning process, and HPO is the problem of choosing a set of optimal hyperparameters for a…

Similar Papers 제목 키워드 기반

TransBoost: A Boosting-Tree Kernel Transfer Learning Algorithm for Improving Financial Inclusion

2021-12-04 · Yiheng Sun, Tian Lu, Cong Wang, Yuan Li 외

The prosperity of mobile and financial technologies has bred and expanded various kinds of financial products to a broader scope of people, which contributes to advocating financial inclusion. It has non-trivial social b…

Transfer Learning

TransBoost: Improving the Best ImageNet Performance using Deep Transduction

2022-05-26 · Omer Belhasin, Guy Bar-Shalom, Ran El-Yaniv

This paper deals with deep transductive learning, and proposes TransBoost as a procedure for fine-tuning any deep neural model to improve its performance on any (unlabeled) test set provided at training time. TransBoost …

image-classificationImage ClassificationTransductive Learning

Improving Multi-fidelity Optimization with a Recurring Learning Rate for Hyperparameter Tuning

2022-09-26 · Hyunjae Lee, Gihyeon Lee, Junhwan Kim, Sungjun Cho 외

Despite the evolution of Convolutional Neural Networks (CNNs), their performance is surprisingly dependent on the choice of hyperparameters. However, it remains challenging to efficiently explore large hyperparameter sea…

image-classificationImage ClassificationTransfer Learning

Warm Starting CMA-ES for Hyperparameter Optimization

2020-12-13 · Masahiro Nomura, Shuhei Watanabe, Youhei Akimoto, Yoshihiko Ozaki 외

Hyperparameter optimization (HPO), formulated as black-box optimization (BBO), is recognized as essential for automation and high performance of machine learning approaches. The CMA-ES is a promising BBO approach with a …

Bayesian OptimizationHyperparameter OptimizationTransfer Learning

Practical Transfer Learning for Bayesian Optimization

2018-02-06 · Matthias Feurer, Benjamin Letham, Frank Hutter, Eytan Bakshy

When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemb…

Bayesian OptimizationGaussian ProcessesHyperparameter OptimizationMeta-Learning+1