HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization
We present a new software, HYPPO, that enables the automatic tuning of hyperparameters of various deep learning (DL) models. Unlike other hyperparameter optimization (HPO) methods, HYPPO uses adaptive surrogate models and directly accounts for uncertainty in model predictions to find accurate and reliable models that make robust predictions. Using asynchronous nested parallelism, we are able to significantly alleviate the computational burden of training complex architectures and quantifying the uncertainty. HYPPO is implemented in Python and can be used with both TensorFlow and PyTorch libraries. We demonstrate various software features on time-series prediction and image classification problems as well as a scientific application in computed tomography image reconstruction. Finally, we show that (1) we can reduce by an order of magnitude the number of evaluations necessary to find the most optimal region in the hyperparameter space and (2) we can reduce by two orders of magnitude the throughput for such HPO process to complete.
Code (0)
등록된 구현이 없습니다.
Tasks
Hyperparameter Optimizationimage-classificationImage ClassificationImage ReconstructionTime SeriesTime Series AnalysisTime Series PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
hyppo: A Multivariate Hypothesis Testing Python Package
We introduce hyppo, a unified library for performing multivariate hypothesis testing, including independence, two-sample, and k-sample testing. While many multivariate independence tests have R packages available, the in…
Two-sample testingHyppo-X: A Scalable Exploratory Framework for Analyzing Complex Phenomics Data
Phenomics is an emerging branch of modern biology that uses high throughput phenotyping tools to capture multiple environmental and phenotypic traits, often at massive spatial and temporal scales. The resulting high dime…
Hyperparameter Optimization of Generative Adversarial Network Models for High-Energy Physics Simulations
The Generative Adversarial Network (GAN) is a powerful and flexible tool that can generate high-fidelity synthesized data by learning. It has seen many applications in simulating events in High Energy Physics (HEP), incl…
Generative Adversarial NetworkHyperparameter OptimizationEnergyLens: Interpretable Closed-Form Energy Models for Multimodal LLM Inference Serving
As large language models span dense, mixture-of-experts, and state-space architectures and are deployed on heterogeneous accelerators under increasingly diverse multimodal workloads, optimising inference energy has becom…
pySOT and POAP: An event-driven asynchronous framework for surrogate optimization
This paper describes Plumbing for Optimization with Asynchronous Parallelism (POAP) and the Python Surrogate Optimization Toolbox (pySOT). POAP is an event-driven framework for building and combining asynchronous optimiz…
Bayesian Optimizationglobal-optimization