Efficient Hyperparameter Optimization for LLM Reinforcement Learning

2026-06-02 · Minping Chen, Bowen Xiao, Du Liang, Chuxuan Zeng, Zeyi Wen arxiv

Reinforcement learning (RL) for large language models (LLMs) is highly sensitive to hyperparameter configurations, making hyperparameter optimization (HPO) essential yet computationally expensive. Existing multi-fidelity HPO methods remain inefficient for LLM RL due to the massive model scale and resource-intensive training cycles. In this paper, we propose Joint Fidelity Hyperparameter Optimization (JF-HPO), which simultaneously adapts both model size and training budget as fidelity. JF-HPO is empowered by: (i) it leverages a small proxy model of the target LLM for efficient training and evaluation in each HPO trial; (ii) it integrates carefully designed early-stopping strategies based on training dynamics; (iii) it introduces an efficient checkpointing mechanism to eliminate redundant computations. Compared with existing HPO methods, JF-HPO significantly improves the computational efficiency of each trial (up to 14.9 times), while achieving better or competitive predictive accuracy under the same time budget. Notably, compared with utilizing hyperparameter configurations from the VeRL Recipe, JF-HPO delivers performance improvements ranging from 5.8% to 111.6%.

📄 PDF Abstract BibTeX arXiv:2606.03073

Code (0)

등록된 구현이 없습니다.

Tasks

Hyperparameter Optimization Computational Efficiency Reinforcement Learning

Quantity vs. Quality: On Hyperparameter Optimization for Deep Reinforcement Learning

2020-07-29 · Lars Hertel, Pierre Baldi, Daniel L. Gillen

Reinforcement learning algorithms can show strong variation in performance between training runs with different random seeds. In this paper we explore how this affects hyperparameter optimization when the goal is to find…

Bayesian Optimization Deep Reinforcement Learning Hyperparameter Optimization reinforcement-learning+2

HyperController: A Hyperparameter Controller for Fast and Stable Training of Reinforcement Learning Neural Networks

2025-04-27 · Jonathan Gornet, Yiannis Kantaros, Bruno Sinopoli

We introduce Hyperparameter Controller (HyperController), a computationally efficient algorithm for hyperparameter optimization during training of reinforcement learning neural networks. HyperController optimizes hyperpa…

Hyperparameter Optimization reinforcement-learning Reinforcement Learning

Hyperparameter Optimization for Driving Strategies Based on Reinforcement Learning

2024-07-19 · Nihal Acharya Adde, Hanno Gottschalk, Andreas Ebert

This paper focuses on hyperparameter optimization for autonomous driving strategies based on Reinforcement Learning. We provide a detailed description of training the RL agent in a simulation environment. Subsequently, w…

Autonomous Driving Bayesian Optimization global-optimization Hyperparameter Optimization+2

Hyperparameter Optimization for Multi-Objective Reinforcement Learning

2023-10-25 · Florian Felten, Daniel Gareev, El-Ghazali Talbi, Grégoire Danoy

Reinforcement learning (RL) has emerged as a powerful approach for tackling complex problems. The recent introduction of multi-objective reinforcement learning (MORL) has further expanded the scope of RL by enabling agen…

Hyperparameter Optimization Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Metaoptimization on a Distributed System for Deep Reinforcement Learning

2019-02-07 · Greg Heinrich, Iuri Frosio

Training intelligent agents through reinforcement learning is a notoriously unstable procedure. Massive parallelization on GPUs and distributed systems has been exploited to generate a large amount of training experience…

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1