Quadruply Stochastic Gaussian Processes
We introduce a stochastic variational inference procedure for training scalable Gaussian process (GP) models whose per-iteration complexity is independent of both the number of training points, $n$, and the number basis functions used in the kernel approximation, $m$. Our central contributions include an unbiased stochastic estimator of the evidence lower bound (ELBO) for a Gaussian likelihood, as well as a stochastic estimator that lower bounds the ELBO for several other likelihoods such as Laplace and logistic. Independence of the stochastic optimization update complexity on $n$ and $m$ enables inference on huge datasets using large capacity GP models. We demonstrate accurate inference on large classification and regression datasets using GPs and relevance vector machines with up to $m = 10^7$ basis functions.
Code (1)
Tasks
Gaussian ProcessesregressionStochastic OptimizationVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Quadruply Stochastic Gradients for Large Scale Nonlinear Semi-Supervised AUC Optimization
Semi-supervised learning is pervasive in real-world applications, where only a few labeled data are available and large amounts of instances remain unlabeled. Since AUC is an important model evaluation metric in classifi…
Stochastic OptimizationThe anatomy of a quadruply imaged gravitational lens system
The key to using a strong gravitational lens system to measure the Hubble constant is to obtain an accurate model of the lens potential. In this paper, we investigate the properties of gravitational lens B1608+656, a qua…
AnatomyvalidImproving Output Uncertainty Estimation and Generalization in Deep Learning via Neural Network Gaussian Processes
We propose a simple method that combines neural networks and Gaussian processes. The proposed method can estimate the uncertainty of outputs and flexibly adjust target functions where training data exist, which are advan…
Gaussian ProcessesVariational InferenceScale Mixtures of Neural Network Gaussian Processes
Recent works have revealed that infinitely-wide feed-forward or recurrent neural networks of any architecture correspond to Gaussian processes referred to as Neural Network Gaussian Processes (NNGPs). While these works h…
Gaussian ProcessesQuadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization
Semi-supervised ordinal regression (S$^2$OR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances remain unlabeled. Recent researches have shown that di…
regression