Asynchronous Distributed Variational Gaussian Processes for Regression
Gaussian processes (GPs) are powerful non-parametric function estimators. However, their applications are largely limited by the expensive computational cost of the inference procedures. Existing stochastic or distributed synchronous variational inferences, although have alleviated this issue by scaling up GPs to millions of samples, are still far from satisfactory for real-world large applications, where the data sizes are often orders of magnitudes larger, say, billions. To solve this problem, we propose ADVGP, the first Asynchronous Distributed Variational Gaussian Process inference for regression, on the recent large-scale machine learning platform, PARAMETERSERVER. ADVGP uses a novel, flexible variational framework based on a weight space augmentation, and implements the highly efficient, asynchronous proximal gradient optimization. While maintaining comparable or better predictive performance, ADVGP greatly improves upon the efficiency of the existing variational methods. With ADVGP, we effortlessly scale up GP regression to a real-world application with billions of samples and demonstrate an excellent, superior prediction accuracy to the popular linear models.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Asynchronous Distributed Gaussian Process Regression for Online Learning and Dynamical Systems: Complementary Document
This is a complementary document for the paper titled "Asynchronous Distributed Gaussian Process Regression for Online Learning and Dynamical Systems".
regressionRecyclable Gaussian Processes
We present a new framework for recycling independent variational approximations to Gaussian processes. The main contribution is the construction of variational ensembles given a dictionary of fitted Gaussian processes wi…
Gaussian ProcessesregressionDistributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models
Gaussian processes (GPs) are a powerful tool for probabilistic inference over functions. They have been applied to both regression and non-linear dimensionality reduction, and offer desirable properties such as uncertain…
Dimensionality ReductionGaussian ProcessesregressionVariational InferenceDistributed Gaussian Processes
To scale Gaussian processes (GPs) to large data sets we introduce the robust Bayesian Committee Machine (rBCM), a practical and scalable product-of-experts model for large-scale distributed GP regression. Unlike state-of…
FormGaussian ProcessesregressionOn Distributed and Asynchronous Sampling of Gaussian Processes for Sequential Binary Hypothesis Testing
In this work, we consider a binary sequential hypothesis testing problem with distributed and asynchronous measurements. The aim is to analyze the effect of sampling times of jointly $\textit{wide-sense stationary}$ (WSS…
Gaussian Processes