Optuna: A Next-generation Hyperparameter Optimization Framework
The purpose of this study is to introduce new design-criteria for next-generation hyperparameter optimization software. The criteria we propose include (1) define-by-run API that allows users to construct the parameter search space dynamically, (2) efficient implementation of both searching and pruning strategies, and (3) easy-to-setup, versatile architecture that can be deployed for various purposes, ranging from scalable distributed computing to light-weight experiment conducted via interactive interface. In order to prove our point, we will introduce Optuna, an optimization software which is a culmination of our effort in the development of a next generation optimization software. As an optimization software designed with define-by-run principle, Optuna is particularly the first of its kind. We will present the design-techniques that became necessary in the development of the software that meets the above criteria, and demonstrate the power of our new design through experimental results and real world applications. Our software is available under the MIT license (https://github.com/pfnet/optuna/).
Code (11)
Tasks
Distributed ComputingHyperparameter OptimizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Optuna Constrained Tree-Structured Parzen Estimator Is a Joint Density Generalization of c-TPE
Constrained hyperparameter optimization (HPO) is common in practice, yet Optuna's widely used constrained TPE lacks algorithmic analysis. While c-TPE proposes an expected constrained improvement (ECI) approach assuming i…
Hyperparameter OptimizationOptuna vs Code Llama: Are LLMs a New Paradigm for Hyperparameter Tuning?
Optimal hyperparameter selection is critical for maximizing neural network performance, especially as models grow in complexity. This work investigates the viability of leveraging large language models (LLMs) for hyperpa…
Computational EfficiencyHyperparameter OptimizationOptunaHub: A Platform for Black-Box Optimization
Black-box optimization (BBO) underpins advances in domains such as AutoML and Materials Informatics, yet implementations of algorithms and benchmarks remain fragmented across research communities. We introduce OptunaHub …
Interim Report on Human-Guided Adaptive Hyperparameter Optimization with Multi-Fidelity Sprints
This case study applies a phased hyperparameter optimization process to compare multitask natural language model variants that utilize multiphase learning rate scheduling and optimizer parameter grouping. We employ short…
Bayesian OptimizationHyperparameter OptimizationJoint Entity and Relation ExtractionLanguage Modeling+3Massively Parallel Genetic Optimization through Asynchronous Propagation of Populations
We present Propulate, an evolutionary optimization algorithm and software package for global optimization and in particular hyperparameter search. For efficient use of HPC resources, Propulate omits the synchronization a…
global-optimization