Optimal Designs of Gaussian Processes with Budgets for Hyperparameter Optimization

2021-01-01 · Yimin Huang, YuJun Li, Zhenguo Li, Zhihua Zhang

The remarkable performance of modern deep learning methods depends critically on the optimization of their hyperparameters. One major challenge is that evaluating a single hyperparameter configuration on large datasets could nowadays easily exceed hours or days. For efficient sampling and fast evaluation, some previous works presented effective computing resource allocation schemes and built a Bayesian surrogate model to sample candidate hyperparameters. However, the model itself is not related to budgets which are set manually. To deal with this problem, a new Gaussian Process model involved in budgets is proposed. Further, for this model, an optimal design is constructed by the equivalence theorem to replace random search as an initial sampling strategy in the search space. Experiments demonstrate that the new model has the best performance among competing methods. Moreover, comparisons between different initial designs with the same model show the advantage of the proposed optimal design.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Gaussian Processes Hyperparameter Optimization

Methods 이 논문이 사용한 방법론

Random Search Random Search replaces the exhaustive enumeration of all combinations by selecting them randomly. This can be simply applied to the discrete setting described above, but also…

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Optimizing Neural Network Hyperparameters with Gaussian Processes for Dialog Act Classification

2016-09-27 · Franck Dernoncourt, Ji Young Lee

Systems based on artificial neural networks (ANNs) have achieved state-of-the-art results in many natural language processing tasks. Although ANNs do not require manually engineered features, ANNs have many hyperparamete…

Bayesian Optimization Dialog Act Classification Gaussian Processes General Classification

Multi-level CNN for lung nodule classification with Gaussian Process assisted hyperparameter optimization

2019-01-02 · Miao Zhang, Huiqi Li, Juan Lyu, Sai Ho Ling 외

This paper investigates lung nodule classification by using deep neural networks (DNNs). Hyperparameter optimization in DNNs is a computationally expensive problem, where evaluating a hyperparameter configuration may tak…

Bayesian Optimization Gaussian Processes General Classification Hyperparameter Optimization+1

Informed Initialization for Bayesian Optimization and Active Learning

2025-10-27 · Carl Hvarfner, David Eriksson, Eytan Bakshy, Max Balandat arxiv

Bayesian Optimization is a widely used method for optimizing expensive black-box functions, relying on probabilistic surrogate models such as Gaussian Processes. The quality of the surrogate model is crucial for good opt…

Gaussian Processes Active Learning

Bayesian Active Learning with Fully Bayesian Gaussian Processes

2022-05-20 · Christoffer Riis, Francisco Antunes, Frederik Boe Hüttel, Carlos Lima Azevedo 외

The bias-variance trade-off is a well-known problem in machine learning that only gets more pronounced the less available data there is. In active learning, where labeled data is scarce or difficult to obtain, neglecting…

Active Learning Gaussian Processes

BOASF: A Unified Framework for Speeding up Automatic Machine Learning via Adaptive Successive Filtering

2025-07-28 · Guanghui Zhu, Xin Fang, Feng Cheng, Lei Wang 외 arxiv

Machine learning has been making great success in many application areas. However, for the non-expert practitioners, it is always very challenging to address a machine learning task successfully and efficiently. Finding …

Hyperparameter Optimization