Hyper-Parameter Tuning for the (1+(λ,λ)) GA
It is known that the $(1+(\lambda,\lambda))$~Genetic Algorithm (GA) with self-adjusting parameter choices achieves a linear expected optimization time on OneMax if its hyper-parameters are suitably chosen. However, it is not very well understood how the hyper-parameter settings influences the overall performance of the $(1+(\lambda,\lambda))$~GA. Analyzing such multi-dimensional dependencies precisely is at the edge of what running time analysis can offer. To make a step forward on this question, we present an in-depth empirical study of the self-adjusting $(1+(\lambda,\lambda))$~GA and its hyper-parameters. We show, among many other results, that a 15\% reduction of the average running time is possible by a slightly different setup, which allows non-identical offspring population sizes of mutation and crossover phase, and more flexibility in the choice of mutation rate and crossover bias --a generalization which may be of independent interest. We also show indication that the parametrization of mutation rate and crossover bias derived by theoretical means for the static variant of the $(1+(\lambda,\lambda))$~GA extends to the non-static case.
Code (1)
Similar Papers 제목 키워드 기반
Tuning the Tuner: Introducing Hyperparameter Optimization for Auto-Tuning
Automatic performance tuning (auto-tuning) is widely used to optimize performance-critical applications across many scientific domains by finding the best program variant among many choices. Efficient optimization algori…
Hyperparameter OptimizationHyperTuning: Toward Adapting Large Language Models without Back-propagation
Fine-tuning large language models for different tasks can be costly and inefficient, and even methods that reduce the number of tuned parameters still require full gradient-based optimization. We propose HyperTuning, a n…
Language ModelingLanguage Modellingparameter-efficient fine-tuningOn Hyper-parameter Tuning for Stochastic Optimization Algorithms
This paper proposes the first-ever algorithmic framework for tuning hyper-parameters of stochastic optimization algorithm based on reinforcement learning. Hyper-parameters impose significant influences on the performance…
Bayesian OptimizationEvolutionary AlgorithmsReinforcement LearningStochastic OptimizationImportance of Tuning Hyperparameters of Machine Learning Algorithms
The performance of many machine learning algorithms depends on their hyperparameter settings. The goal of this study is to determine whether it is important to tune a hyperparameter or whether it can be safely set to a d…
BIG-bench Machine LearningSurrogate Model Based Hyperparameter Tuning for Deep Learning with SPOT
A surrogate model based hyperparameter tuning approach for deep learning is presented. This article demonstrates how the architecture-level parameters (hyperparameters) of deep learning models that were implemented in Ke…
Deep Learning