Bayesian Hyperparameter Optimization for Ensemble Learning
In this paper, we bridge the gap between hyperparameter optimization and ensemble learning by performing Bayesian optimization of an ensemble with regards to its hyperparameters. Our method consists in building a fixed-size ensemble, optimizing the configuration of one classifier of the ensemble at each iteration of the hyperparameter optimization algorithm, taking into consideration the interaction with the other models when evaluating potential performances. We also consider the case where the ensemble is to be reconstructed at the end of the hyperparameter optimization phase, through a greedy selection over the pool of models generated during the optimization. We study the performance of our proposed method on three different hyperparameter spaces, showing that our approach is better than both the best single model and a greedy ensemble construction over the models produced by a standard Bayesian optimization.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationEnsemble LearningHyperparameter OptimizationSimilar Papers 제목 키워드 기반
Practical Transfer Learning for Bayesian Optimization
When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemb…
Bayesian OptimizationGaussian ProcessesHyperparameter OptimizationMeta-Learning+1Deep-Ensemble-Based Uncertainty Quantification in Spatiotemporal Graph Neural Networks for Traffic Forecasting
Deep-learning-based data-driven forecasting methods have produced impressive results for traffic forecasting. A major limitation of these methods, however, is that they provide forecasts without estimates of uncertainty,…
Bayesian OptimizationHyperparameter OptimizationUncertainty QuantificationFast Hyperparameter Optimization of Deep Neural Networks via Ensembling Multiple Surrogates
The performance of deep neural networks crucially depends on good hyperparameter configurations. Bayesian optimization is a powerful framework for optimizing the hyperparameters of DNNs. These methods need sufficient eva…
Bayesian OptimizationHyperparameter OptimizationOptimizing Ensemble Weights and Hyperparameters of Machine Learning Models for Regression Problems
Aggregating multiple learners through an ensemble of models aim to make better predictions by capturing the underlying distribution of the data more accurately. Different ensembling methods, such as bagging, boosting, an…
BIG-bench Machine LearningregressionClick prediction boosting via Bayesian hyperparameter optimization based ensemble learning pipelines
Online travel agencies (OTA's) advertise their website offers on meta-search bidding engines. The problem of predicting the number of clicks a hotel would receive for a given bid amount is an important step in the manage…
Dimensionality ReductionEnsemble LearningHyperparameter OptimizationManagement