Reducing The Search Space For Hyperparameter Optimization Using Group Sparsity
We propose a new algorithm for hyperparameter selection in machine learning algorithms. The algorithm is a novel modification of Harmonica, a spectral hyperparameter selection approach using sparse recovery methods. In particular, we show that a special encoding of hyperparameter space enables a natural group-sparse recovery formulation, which when coupled with HyperBand (a multi-armed bandit strategy) leads to improvement over existing hyperparameter optimization methods such as Successive Halving and Random Search. Experimental results on image datasets such as CIFAR-10 confirm the benefits of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningHyperparameter OptimizationSimilar Papers 제목 키워드 기반
Bayesian Hyperparameter Optimization with BoTorch, GPyTorch and Ax
Deep learning models are full of hyperparameters, which are set manually before the learning process can start. To find the best configuration for these hyperparameters in such a high dimensional space, with time-consumi…
Bayesian OptimizationGaussian ProcessesGraph ClassificationHyperparameter OptimizationSigOpt Mulch: An Intelligent System for AutoML of Gradient Boosted Trees
Gradient boosted trees (GBTs) are ubiquitous models used by researchers, machine learning (ML) practitioners, and data scientists because of their robust performance, interpretable behavior, and ease-of-use. One critical…
AutoMLHyperparameter OptimizationLearning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
Bayesian optimization (BO) is a successful methodology to optimize black-box functions that are expensive to evaluate. While traditional methods optimize each black-box function in isolation, there has been recent intere…
Bayesian OptimizationHyperparameter OptimizationTransfer LearningHyperopt-Sklearn: Automatic Hyperparameter Configuration for Scikit-Learn
Hyperopt-sklearn is a new software project that provides automatic algorithm configuration of the Scikit-learn machine learning library. Following Auto-Weka, we take the view that the choice of classifier and even the ch…
AutoMLBenchmarkingHyperparameter OptimizationMulti-Objective Optimization of Performance and Interpretability of Tabular Supervised Machine Learning Models
We present a model-agnostic framework for jointly optimizing the predictive performance and interpretability of supervised machine learning models for tabular data. Interpretability is quantified via three measures: feat…
feature selectionHyperparameter Optimization