Sparse within Sparse Gaussian Processes using Neighbor Information
Approximations to Gaussian processes based on inducing variables, combined with variational inference techniques, enable state-of-the-art sparse approaches to infer GPs at scale through mini batch-based learning. In this work, we address one limitation of sparse GPs, which is due to the challenge in dealing with a large number of inducing variables without imposing a special structure on the inducing inputs. In particular, we introduce a novel hierarchical prior, which imposes sparsity on the set of inducing variables. We treat our model variationally, and we experimentally show considerable computational gains compared to standard sparse GPs when sparsity on the inducing variables is realized considering the nearest inducing inputs of a random mini-batch of the data. We perform an extensive experimental validation that demonstrates the effectiveness of our approach compared to the state-of-the-art. Our approach enables the possibility to use sparse GPs using a large number of inducing points without incurring a prohibitive computational cost.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Variational Nearest Neighbor Gaussian Process
Variational approximations to Gaussian processes (GPs) typically use a small set of inducing points to form a low-rank approximation to the covariance matrix. In this work, we instead exploit a sparse approximation of th…
Gaussian ProcessesStochastic OptimizationVariational sparse inverse Cholesky approximation for latent Gaussian processes via double Kullback-Leibler minimization
To achieve scalable and accurate inference for latent Gaussian processes, we propose a variational approximation based on a family of Gaussian distributions whose covariance matrices have sparse inverse Cholesky (SIC) fa…
Gaussian ProcessesDecoupled Sparse Gaussian Processes Components]{Decoupled Sparse Gaussian Processes Components : Separating Decision Making from Data Manifold Fitting
We propose a decoupling in Reproducing Kernel Hilbert Space of the parametric and non-parametric components of Sparse Gaussian Processes. We demonstrate that this decoupling results in a set of inducing points concentra…
Decision MakingGaussian ProcessesregressionVariance State Propagation for Structured Sparse Bayesian Learning
We propose a compressed sensing algorithm termed variance state propagation (VSP) for block-sparse signals, i.e., sparse signals that have nonzero coefficients occurring in clusters. The VSP algorithm is developed under …
compressed sensingVariational zero-inflated Gaussian processes with sparse kernels
Zero-inflated datasets, which have an excess of zero outputs, are commonly encountered in problems such as climate or rare event modelling. Conventional machine learning approaches tend to overestimate the non-zeros lead…
Gaussian ProcessesVariational Inference