Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning
In this paper, we address the problem of \emph{cost-sensitive} hyperparameter optimization (HPO) built upon freeze-thaw Bayesian optimization (BO). Specifically, we assume a scenario where users want to early-stop the HPO process when the expected performance improvement is not satisfactory with respect to the additional computational cost. Motivated by this scenario, we introduce \emph{utility} in the freeze-thaw framework, a function describing the trade-off between the cost and performance that can be estimated from the user's preference data. This utility function, combined with our novel acquisition function and stopping criterion, allows us to dynamically continue training the configuration that we expect to maximally improve the utility in the future, and also automatically stop the HPO process around the maximum utility. Further, we improve the sample efficiency of existing freeze-thaw methods with transfer learning to develop a specialized surrogate model for the cost-sensitive HPO problem. We validate our algorithm on established multi-fidelity HPO benchmarks and show that it outperforms all the previous freeze-thaw BO and transfer-BO baselines we consider, while achieving a significantly better trade-off between the cost and performance. Our code is publicly available at https://github.com/db-Lee/CFBO.
Code (0)
등록된 구현이 없습니다.
Tasks
Hyperparameter OptimizationTransfer LearningSimilar Papers 제목 키워드 기반
In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization
With the increasing computational costs associated with deep learning, automated hyperparameter optimization methods, strongly relying on black-box Bayesian optimization (BO), face limitations. Freeze-thaw BO offers a pr…
Bayesian OptimizationHyperparameter OptimizationIn-Context LearningTune My Adam, Please!
The Adam optimizer remains one of the most widely used optimizers in deep learning, and effectively tuning its hyperparameters is key to optimizing performance. However, tuning can be tedious and costly. Freeze-thaw Baye…
Hyperparameter OptimizationXylem embolism and bubble formation during freezing suggest complex dynamics of pressure-tension in Betula pendula stems
Freeze-thaw-induced embolism is a key limiting factor for perennial plants in frost-exposed environments. Gas bubbles are formed during freezing and expand during thawing resulting in xylem embolism. However, when water …
Freeze-Thaw Bayesian Optimization
In this paper we develop a dynamic form of Bayesian optimization for machine learning models with the goal of rapidly finding good hyperparameter settings. Our method uses the partial information gained during the traini…
Bayesian OptimizationBIG-bench Machine LearningActive Illumination Control in Low-Light Environments using NightHawk
Subterranean environments such as culverts present significant challenges to robot vision due to dim lighting and lack of distinctive features. Although onboard illumination can help, it introduces issues such as specula…
Bayesian Optimization