A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learning

2024-12-10 · Jacob Adkins, Michael Bowling, Adam White

The performance of modern reinforcement learning algorithms critically relies on tuning ever-increasing numbers of hyperparameters. Often, small changes in a hyperparameter can lead to drastic changes in performance, and different environments require very different hyperparameter settings to achieve state-of-the-art performance reported in the literature. We currently lack a scalable and widely accepted approach to characterizing these complex interactions. This work proposes a new empirical methodology for studying, comparing, and quantifying the sensitivity of an algorithm's performance to hyperparameter tuning for a given set of environments. We then demonstrate the utility of this methodology by assessing the hyperparameter sensitivity of several commonly used normalization variants of PPO. The results suggest that several algorithmic performance improvements may, in fact, be a result of an increased reliance on hyperparameter tuning.

📄 PDF Abstract BibTeX arXiv:2412.07165

Code (1)

jadkins99/hyperparameter_sensitivity 공식 구현 jax

Tasks

reinforcement-learning Reinforcement LearningSensitivity

Methods 이 논문이 사용한 방법론

Entropy Regularization 설명 없음

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Sample-Efficient Automated Deep Reinforcement Learning

2020-09-03 · ICLR 2021 1 · Jörg K. H. Franke, Gregor Köhler, André Biedenkapp, Frank Hutter

Despite significant progress in challenging problems across various domains, applying state-of-the-art deep reinforcement learning (RL) algorithms remains challenging due to their sensitivity to the choice of hyperparame…

Deep Reinforcement Learning Hyperparameter Optimization MuJoCo reinforcement-learning+3

Assessing the Effects of Hyperparameters on Knowledge Graph Embedding Quality

2022-07-01 · Oliver Lloyd, Yi Liu, Tom Gaunt

Embedding knowledge graphs into low-dimensional spaces is a popular method for applying approaches, such as link prediction or node classification, to these databases. This embedding process is very costly in terms of bo…

Graph Embedding Knowledge Graph Embedding Knowledge Graphs Link Prediction+2

Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining

2025-05-30 · Weiyi Wang, Junwei Deng, Yuzheng Hu, Shiyuan Zhang 외

Data attribution methods, which quantify the influence of individual training data points on a machine learning model, have gained increasing popularity in data-centric applications in modern AI. Despite a recent surge o…

Sensitivity

Hyperparameter Optimization for Driving Strategies Based on Reinforcement Learning

2024-07-19 · Nihal Acharya Adde, Hanno Gottschalk, Andreas Ebert

This paper focuses on hyperparameter optimization for autonomous driving strategies based on Reinforcement Learning. We provide a detailed description of training the RL agent in a simulation environment. Subsequently, w…

Autonomous Driving Bayesian Optimization global-optimization Hyperparameter Optimization+2

Alpha MAML: Adaptive Model-Agnostic Meta-Learning

2019-05-17 · Harkirat Singh Behl, Atılım Güneş Baydin, Philip H. S. Torr

Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks. The MAML algorithm performs well on fe…

Few-Shot Learning General Classification Meta-Learning model+1