A Study of Genetic Algorithms for Hyperparameter Optimization of Neural Networks in Machine Translation
With neural networks having demonstrated their versatility and benefits, the need for their optimal performance is as prevalent as ever. A defining characteristic, hyperparameters, can greatly affect its performance. Thus engineers go through a process, tuning, to identify and implement optimal hyperparameters. That being said, excess amounts of manual effort are required for tuning network architectures, training configurations, and preprocessing settings such as Byte Pair Encoding (BPE). In this study, we propose an automatic tuning method modeled after Darwin's Survival of the Fittest Theory via a Genetic Algorithm (GA). Research results show that the proposed method, a GA, outperforms a random selection of hyperparameters.
Code (1)
Tasks
Hyperparameter OptimizationMachine TranslationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Genetic algorithm-based hyperparameter optimization of deep learning models for PM2.5 time-series prediction
Since air pollution negatively affects human health and causes serious diseases, accurate air pollution prediction is essential regarding environmental sustainability. Although conventional statistical and machine learni…
Air Pollution PredictionDeep LearningHyperparameter OptimizationTime Series+2Deep Genetic Network
Optimizing a neural network's performance is a tedious and time taking process, this iterative process does not have any defined solution which can work for all the problems. Optimization can be roughly categorized into …
Hyperparameter OptimizationEarly years of Biased Random-Key Genetic Algorithms: A systematic review
This paper presents a systematic literature review and bibliometric analysis focusing on Biased Random-Key Genetic Algorithms (BRKGA). BRKGA is a metaheuristic framework that uses random-key-based chromosomes with biased…
Combinatorial OptimizationSystematic Literature ReviewA Comparative Study of Hyperparameter Tuning Methods
The study emphasizes the challenge of finding the optimal trade-off between bias and variance, especially as hyperparameter optimization increases in complexity. Through empirical analysis, three hyperparameter tuning al…
Hyperparameter OptimizationregressionEfficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm
Convolutional Neural Networks (CNN) have gained great success in many artificial intelligence tasks. However, finding a good set of hyperparameters for a CNN remains a challenging task. It usually takes an expert with de…
CPUHyperparameter Optimization