Hyp-RL : Hyperparameter Optimization by Reinforcement Learning
Hyperparameter tuning is an omnipresent problem in machine learning as it is an integral aspect of obtaining the state-of-the-art performance for any model. Most often, hyperparameters are optimized just by training a model on a grid of possible hyperparameter values and taking the one that performs best on a validation sample (grid search). More recently, methods have been introduced that build a so-called surrogate model that predicts the validation loss for a specific hyperparameter setting, model and dataset and then sequentially select the next hyperparameter to test, based on a heuristic function of the expected value and the uncertainty of the surrogate model called acquisition function (sequential model-based Bayesian optimization, SMBO). In this paper we model the hyperparameter optimization problem as a sequential decision problem, which hyperparameter to test next, and address it with reinforcement learning. This way our model does not have to rely on a heuristic acquisition function like SMBO, but can learn which hyperparameters to test next based on the subsequent reduction in validation loss they will eventually lead to, either because they yield good models themselves or because they allow the hyperparameter selection policy to build a better surrogate model that is able to choose better hyperparameters later on. Experiments on a large battery of 50 data sets demonstrate that our method outperforms the state-of-the-art approaches for hyperparameter learning.
Code (1)
Tasks
Bayesian OptimizationHyperparameter Optimizationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Quantity vs. Quality: On Hyperparameter Optimization for Deep Reinforcement Learning
Reinforcement learning algorithms can show strong variation in performance between training runs with different random seeds. In this paper we explore how this affects hyperparameter optimization when the goal is to find…
Bayesian OptimizationDeep Reinforcement LearningHyperparameter Optimizationreinforcement-learning+2HyperController: A Hyperparameter Controller for Fast and Stable Training of Reinforcement Learning Neural Networks
We introduce Hyperparameter Controller (HyperController), a computationally efficient algorithm for hyperparameter optimization during training of reinforcement learning neural networks. HyperController optimizes hyperpa…
Hyperparameter Optimizationreinforcement-learningReinforcement LearningHyperparameter Optimization for Driving Strategies Based on Reinforcement Learning
This paper focuses on hyperparameter optimization for autonomous driving strategies based on Reinforcement Learning. We provide a detailed description of training the RL agent in a simulation environment. Subsequently, w…
Autonomous DrivingBayesian Optimizationglobal-optimizationHyperparameter Optimization+2Hyperparameter Optimization for Multi-Objective Reinforcement Learning
Reinforcement learning (RL) has emerged as a powerful approach for tackling complex problems. The recent introduction of multi-objective reinforcement learning (MORL) has further expanded the scope of RL by enabling agen…
Hyperparameter OptimizationMulti-Objective Reinforcement Learningreinforcement-learningReinforcement Learning+1Metaoptimization on a Distributed System for Deep Reinforcement Learning
Training intelligent agents through reinforcement learning is a notoriously unstable procedure. Massive parallelization on GPUs and distributed systems has been exploited to generate a large amount of training experience…
Atari GamesDeep Reinforcement Learningreinforcement-learningReinforcement Learning+1