Deep Ranking Ensembles for Hyperparameter Optimization
Automatically optimizing the hyperparameters of Machine Learning algorithms is one of the primary open questions in AI. Existing work in Hyperparameter Optimization (HPO) trains surrogate models for approximating the response surface of hyperparameters as a regression task. In contrast, we hypothesize that the optimal strategy for training surrogates is to preserve the ranks of the performances of hyperparameter configurations as a Learning to Rank problem. As a result, we present a novel method that meta-learns neural network surrogates optimized for ranking the configurations' performances while modeling their uncertainty via ensembling. In a large-scale experimental protocol comprising 12 baselines, 16 HPO search spaces and 86 datasets/tasks, we demonstrate that our method achieves new state-of-the-art results in HPO.
Code (0)
등록된 구현이 없습니다.
Tasks
Hyperparameter OptimizationLearning-To-RankMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Optimizing Ensemble Weights and Hyperparameters of Machine Learning Models for Regression Problems
Aggregating multiple learners through an ensemble of models aim to make better predictions by capturing the underlying distribution of the data more accurately. Different ensembling methods, such as bagging, boosting, an…
BIG-bench Machine LearningregressionMulti-Objective Hyperparameter Tuning and Feature Selection using Filter Ensembles
Both feature selection and hyperparameter tuning are key tasks in machine learning. Hyperparameter tuning is often useful to increase model performance, while feature selection is undertaken to attain sparse models. Spar…
feature selectionHyperparameter OptimizationHyperparameter Ensembles for Robustness and Uncertainty Quantification
Ensembles over neural network weights trained from different random initialization, known as deep ensembles, achieve state-of-the-art accuracy and calibration. The recently introduced batch ensembles provide a drop-in re…
Diversityimage-classificationImage ClassificationUncertainty QuantificationShort-answer scoring with ensembles of pretrained language models
We investigate the effectiveness of ensembles of pretrained transformer-based language models on short answer questions using the Kaggle Automated Short Answer Scoring dataset. We fine-tune a collection of popular small,…
Hyperparameter OptimizationSequential Model-Based Ensemble Optimization
One of the most tedious tasks in the application of machine learning is model selection, i.e. hyperparameter selection. Fortunately, recent progress has been made in the automation of this process, through the use of seq…
modelModel Selection