Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020
This paper presents the results and insights from the black-box optimization (BBO) challenge at NeurIPS 2020 which ran from July-October, 2020. The challenge emphasized the importance of evaluating derivative-free optimizers for tuning the hyperparameters of machine learning models. This was the first black-box optimization challenge with a machine learning emphasis. It was based on tuning (validation set) performance of standard machine learning models on real datasets. This competition has widespread impact as black-box optimization (e.g., Bayesian optimization) is relevant for hyperparameter tuning in almost every machine learning project as well as many applications outside of machine learning. The final leaderboard was determined using the optimization performance on held-out (hidden) objective functions, where the optimizers ran without human intervention. Baselines were set using the default settings of several open-source black-box optimization packages as well as random search.
Code (1)
Tasks
Bayesian OptimizationBIG-bench Machine LearningSimilar Papers 제목 키워드 기반
Computer-aided diagnosis of lung nodule using gradient tree boosting and Bayesian optimization
We aimed to evaluate computer-aided diagnosis (CADx) system for lung nodule classification focusing on (i) usefulness of gradient tree boosting (XGBoost) and (ii) effectiveness of parameter optimization using Bayesian op…
Bayesian OptimizationLung Nodule ClassificationScalable Differentially Private Bayesian Optimization
In recent years, there has been much work on scaling Bayesian Optimization to high-dimensional problems, for example hyperparameter tuning in large machine learning models. These scalable methods have been successful, fi…
Bayesian OptimizationA Modified Bayesian Optimization based Hyper-Parameter Tuning Approach for Extreme Gradient Boosting
It is already reported in the literature that the performance of a machine learning algorithm is greatly impacted by performing proper Hyper-Parameter optimization. One of the ways to perform Hyper-Parameter optimization…
Bayesian OptimizationBlack-Box Optimization in Machine Learning with Trust Region Based Derivative Free Algorithm
In this work, we utilize a Trust Region based Derivative Free Optimization (DFO-TR) method to directly maximize the Area Under Receiver Operating Characteristic Curve (AUC), which is a nonsmooth, noisy function. We show …
Bayesian OptimizationBIG-bench Machine LearningText-to-Image GenerationLearning to Warm-Start Bayesian Hyperparameter Optimization
Hyperparameter optimization aims to find the optimal hyperparameter configuration of a machine learning model, which provides the best performance on a validation dataset. Manual search usually leads to get stuck in a lo…
Bayesian Optimizationglobal-optimizationHyperparameter OptimizationMetric Learning