Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations
Gaussian processes (GP) are Bayesian non-parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP regression methods that exploit low-rank covariance matrix approximations for distributing the computational load among parallel machines to achieve time efficiency and scalability. We theoretically guarantee the predictive performances of our proposed parallel GPs to be equivalent to that of some centralized approximate GP regression methods: The computation of their centralized counterparts can be distributed among parallel machines, hence achieving greater time efficiency and scalability. We analytically compare the properties of our parallel GPs such as time, space, and communication complexity. Empirical evaluation on two real-world datasets in a cluster of 20 computing nodes shows that our parallel GPs are significantly more time-efficient and scalable than their centralized counterparts and exact/full GP while achieving predictive performances comparable to full GP.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesregressionSimilar Papers 제목 키워드 기반
Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations
Gaussian processes (GP) are Bayesian non-parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time co…
Gaussian ProcessesregressionIntrinsic Wrapped Gaussian Process Regression Modeling for Manifold-valued Response Variable
In this paper, we propose a novel intrinsic wrapped Gaussian process regression model for response variable measured on Riemannian manifold. We apply the parallel transport operator to define an intrinsic covariance stru…
regressionTensor Regression Meets Gaussian Processes
Low-rank tensor regression, a new model class that learns high-order correlation from data, has recently received considerable attention. At the same time, Gaussian processes (GP) are well-studied machine learning models…
Bayesian InferenceGaussian ProcessesregressionEmbarrassingly Parallel Inference for Gaussian Processes
Training Gaussian process-based models typically involves an $ O(N^3)$ computational bottleneck due to inverting the covariance matrix. Popular methods for overcoming this matrix inversion problem cannot adequately model…
Gaussian ProcessesMixture-of-ExpertsregressionGaussian Process Regression under Computational and Epistemic Misspecification
Gaussian process regression is a classical kernel method for function estimation and data interpolation. In large data applications, computational costs can be reduced using low-rank or sparse approximations of the kerne…
regression