Scalable Gaussian Process Hyperparameter Optimization via Coverage Regularization
Gaussian processes (GPs) are Bayesian non-parametric models popular in a variety of applications due to their accuracy and native uncertainty quantification (UQ). Tuning GP hyperparameters is critical to ensure the validity of prediction accuracy and uncertainty; uniquely estimating multiple hyperparameters in, e.g. the Matern kernel can also be a significant challenge. Moreover, training GPs on large-scale datasets is a highly active area of research: traditional maximum likelihood hyperparameter training requires quadratic memory to form the covariance matrix and has cubic training complexity. To address the scalable hyperparameter tuning problem, we present a novel algorithm which estimates the smoothness and length-scale parameters in the Matern kernel in order to improve robustness of the resulting prediction uncertainties. Using novel loss functions similar to those in conformal prediction algorithms in the computational framework provided by the hyperparameter estimation algorithm MuyGPs, we achieve improved UQ over leave-one-out likelihood maximization while maintaining a high degree of scalability as demonstrated in numerical experiments.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionGaussian ProcessesHyperparameter OptimizationPredictionUncertainty QuantificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Scalable Hyperparameter Optimization with Products of Gaussian Process Experts
In machine learning, hyperparameter optimization is a challenging but necessary task that is usually approached in a computationally expensive manner such as grid-search. Out of this reason, surrogate based black-box opt…
Gaussian ProcessesHyperparameter OptimizationPreconditioning for Scalable Gaussian Process Hyperparameter Optimization
Gaussian process hyperparameter optimization requires linear solves with, and log-determinants of, large kernel matrices. Iterative numerical techniques are becoming popular to scale to larger datasets, relying on the co…
Gaussian ProcessesHyperparameter OptimizationBias-Free Scalable Gaussian Processes via Randomized Truncations
Scalable Gaussian Process methods are computationally attractive, yet introduce modeling biases that require rigorous study. This paper analyzes two common techniques: early truncated conjugate gradients (CG) and random …
Gaussian ProcessesHyperparameter Optimization for Driving Strategies Based on Reinforcement Learning
This paper focuses on hyperparameter optimization for autonomous driving strategies based on Reinforcement Learning. We provide a detailed description of training the RL agent in a simulation environment. Subsequently, w…
Autonomous DrivingBayesian Optimizationglobal-optimizationHyperparameter Optimization+2Scalable Hyperparameter Optimization with Lazy Gaussian Processes
Most machine learning methods require careful selection of hyper-parameters in order to train a high performing model with good generalization abilities. Hence, several automatic selection algorithms have been introduced…
Bayesian OptimizationGaussian ProcessesHyperparameter Optimization