Tighter sparse variational Gaussian processes
Sparse variational Gaussian process (GP) approximations based on inducing points have become the de facto standard for scaling GPs to large datasets, owing to their theoretical elegance, computational efficiency, and ease of implementation. This paper introduces a provably tighter variational approximation by relaxing the standard assumption that the conditional approximate posterior given the inducing points must match that in the prior. The key innovation is to modify the conditional posterior to have smaller variances than that of the prior at the training points. We derive the collapsed bound for the regression case, describe how to use the proposed approximation in large data settings, and discuss its application to handle orthogonally structured inducing points and GP latent variable models. Extensive experiments on regression benchmarks, classification, and latent variable models demonstrate that the proposed approximation consistently matches or outperforms standard sparse variational GPs while maintaining the same computational cost. An implementation will be made available in all popular GP packages.
Code (0)
등록된 구현이 없습니다.
Tasks
Computational EfficiencyGaussian ProcessesregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sparse Orthogonal Variational Inference for Gaussian Processes
We introduce a new interpretation of sparse variational approximations for Gaussian processes using inducing points, which can lead to more scalable algorithms than previous methods. It is based on decomposing a Gaussian…
Gaussian ProcessesMulti-class ClassificationregressionVariational InferenceDual Parameterization of Sparse Variational Gaussian Processes
Sparse variational Gaussian process (SVGP) methods are a common choice for non-conjugate Gaussian process inference because of their computational benefits. In this paper, we improve their computational efficiency by usi…
Computational EfficiencyGaussian ProcessesNew Bounds for Sparse Variational Gaussian Processes
Sparse variational Gaussian processes (GPs) construct tractable posterior approximations to GP models. At the core of these methods is the assumption that the true posterior distribution over training function values ${\…
Gaussian ProcessesStochastic OptimizationTighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradients
We propose a lower bound on the log marginal likelihood of Gaussian process regression models that can be computed without matrix factorisation of the full kernel matrix. We show that approximate maximum likelihood learn…
Gaussian ProcessesregressionVariational Inference for Sparse Gaussian Process Modulated Hawkes Process
The Hawkes process (HP) has been widely applied to modeling self-exciting events including neuron spikes, earthquakes and tweets. To avoid designing parametric triggering kernel and to be able to quantify the prediction …
Model OptimizationModel SelectionVariational Inference