The Impact of Hyper-Parameter Tuning for Landscape-Aware Performance Regression and Algorithm Selection
Automated algorithm selection and configuration methods that build on exploratory landscape analysis (ELA) are becoming very popular in Evolutionary Computation. However, despite a significantly growing number of applications, the underlying machine learning models are often chosen in an ad-hoc manner. We show in this work that three classical regression methods are able to achieve meaningful results for ELA-based algorithm selection. For those three models -- random forests, decision trees, and bagging decision trees -- the quality of the regression models is highly impacted by the chosen hyper-parameters. This has significant effects also on the quality of the algorithm selectors that are built on top of these regressions. By comparing a total number of 30 different models, each coupled with 2 complementary regression strategies, we derive guidelines for the tuning of the regression models and provide general recommendations for a more systematic use of classical machine learning models in landscape-aware algorithm selection. We point out that a choice of the machine learning model merits to be carefully undertaken and further investigated.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningregressionSimilar Papers 제목 키워드 기반
A Three-regime Model of Network Pruning
Recent work has highlighted the complex influence training hyperparameters, e.g., the number of training epochs, can have on the prunability of machine learning models. Perhaps surprisingly, a systematic approach to pred…
Efficient Neural NetworkHyperparameter OptimizationmodelNetwork PruningAutoRL Hyperparameter Landscapes
Although Reinforcement Learning (RL) has shown to be capable of producing impressive results, its use is limited by the impact of its hyperparameters on performance. This often makes it difficult to achieve good results …
AutoMLHyperparameter OptimizationOpen-Ended Question AnsweringReinforcement Learning (RL)Explainable Landscape-Aware Optimization Performance Prediction
Efficient solving of an unseen optimization problem is related to appropriate selection of an optimization algorithm and its hyper-parameters. For this purpose, automated algorithm performance prediction should be perfor…
PredictionCompleted Hyperparameter Transfer across Modules, Width, Depth, Batch and Duration
Hyperparameter tuning can dramatically impact training stability and final performance of large-scale models. Recent works on neural network parameterisations, such as $μ$P, have enabled transfer of optimal global hyperp…
Architecture-Aware Learning Curve Extrapolation via Graph Ordinary Differential Equation
Learning curve extrapolation predicts neural network performance from early training epochs and has been applied to accelerate AutoML, facilitating hyperparameter tuning and neural architecture search. However, existing …
AutoMLNeural Architecture Search