Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition
We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We propose the harmonic kernel decomposition (HKD), which uses Fourier series to decompose a kernel as a sum of orthogonal kernels. Our variational approximation exploits this orthogonality to enable a large number of inducing points at a low computational cost. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections, and it significantly outperforms standard variational methods in scalability and accuracy. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
Code (2)
Tasks
Gaussian ProcessesregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sparse Gaussian Processes with Spherical Harmonic Features Revisited
We revisit the Gaussian process model with spherical harmonic features and study connections between the associated RKHS, its eigenstructure and deep models. Based on this, we introduce a new class of kernels which corre…
Gaussian ProcessesScaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes
Automating statistical modelling is a challenging problem in artificial intelligence. The Automatic Statistician takes a first step in this direction, by employing a kernel search algorithm with Gaussian Processes (GP) t…
Gaussian ProcessesModel SelectionregressionVariational zero-inflated Gaussian processes with sparse kernels
Zero-inflated datasets, which have an excess of zero outputs, are commonly encountered in problems such as climate or rare event modelling. Conventional machine learning approaches tend to overestimate the non-zeros lead…
Gaussian ProcessesVariational InferenceStochastic Variational Deep Kernel Learning
Deep kernel learning combines the non-parametric flexibility of kernel methods with the inductive biases of deep learning architectures. We propose a novel deep kernel learning model and stochastic variational inference …
Gaussian ProcessesGeneral ClassificationMulti-Task LearningVariational InferenceSparse Gaussian Processes with Spherical Harmonic Features
We introduce a new class of inter-domain variational Gaussian processes (GP) where data is mapped onto the unit hypersphere in order to use spherical harmonic representations. Our inference scheme is comparable to variat…
Gaussian Processes