Optimizing Neural Network Hyperparameters with Gaussian Processes for Dialog Act Classification
Systems based on artificial neural networks (ANNs) have achieved state-of-the-art results in many natural language processing tasks. Although ANNs do not require manually engineered features, ANNs have many hyperparameters to be optimized. The choice of hyperparameters significantly impacts models' performances. However, the ANN hyperparameters are typically chosen by manual, grid, or random search, which either requires expert experiences or is computationally expensive. Recent approaches based on Bayesian optimization using Gaussian processes (GPs) is a more systematic way to automatically pinpoint optimal or near-optimal machine learning hyperparameters. Using a previously published ANN model yielding state-of-the-art results for dialog act classification, we demonstrate that optimizing hyperparameters using GP further improves the results, and reduces the computational time by a factor of 4 compared to a random search. Therefore it is a useful technique for tuning ANN models to yield the best performances for natural language processing tasks.
Code (1)
Tasks
Bayesian OptimizationDialog Act ClassificationGaussian ProcessesGeneral ClassificationSimilar Papers 제목 키워드 기반
Bayesian autoregression to optimize temporal Matérn kernel Gaussian process hyperparameters
Gaussian processes are important models in the field of probabilistic numerics. We present a procedure for optimizing Matérn kernel temporal Gaussian processes with respect to the kernel covariance function's hyperparame…
Gaussian ProcessesGaussian Process Classification with Privileged Information by Soft-to-Hard Labeling Transfer
Learning using privileged information is an attractive problem setting that helps many learning scenarios in the real world. A state-of-the-art method of Gaussian process classification (GPC) with privileged information …
Gaussian ProcessesGeneral ClassificationTransfer LearningGaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applications
Gaussian processes have become a promising tool for various safety-critical settings, since the posterior variance can be used to directly estimate the model error and quantify risk. However, state-of-the-art techniques …
Gaussian ProcessesBayesian Hyperparameter Optimization with BoTorch, GPyTorch and Ax
Deep learning models are full of hyperparameters, which are set manually before the learning process can start. To find the best configuration for these hyperparameters in such a high dimensional space, with time-consumi…
Bayesian OptimizationGaussian ProcessesGraph ClassificationHyperparameter OptimizationFully Bayesian Differential Gaussian Processes through Stochastic Differential Equations
Traditional deep Gaussian processes model the data evolution using a discrete hierarchy, whereas differential Gaussian processes (DIFFGPs) represent the evolution as an infinitely deep Gaussian process. However, prior DI…
Bayesian InferenceGaussian Processes