Scalable Multi-Class Gaussian Process Classification using Expectation Propagation
This paper describes an expectation propagation (EP) method for multi-class classification with Gaussian processes that scales well to very large datasets. In such a method the estimate of the log-marginal-likelihood involves a sum across the data instances. This enables efficient training using stochastic gradients and mini-batches. When this type of training is used, the computational cost does not depend on the number of data instances $N$. Furthermore, extra assumptions in the approximate inference process make the memory cost independent of $N$. The consequence is that the proposed EP method can be used on datasets with millions of instances. We compare empirically this method with alternative approaches that approximate the required computations using variational inference. The results show that it performs similar or even better than these techniques, which sometimes give significantly worse predictive distributions in terms of the test log-likelihood. Besides this, the training process of the proposed approach also seems to converge in a smaller number of iterations.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGaussian ProcessesGeneral ClassificationMulti-class ClassificationVariational InferenceSimilar Papers 제목 키워드 기반
Scalable Gaussian Process Classification with Additive Noise for Various Likelihoods
Gaussian process classification (GPC) provides a flexible and powerful statistical framework describing joint distributions over function space. Conventional GPCs however suffer from (i) poor scalability for big data due…
ClassificationGeneral ClassificationGPRMulti-class Classification+1Scalable Cross Validation Losses for Gaussian Process Models
We introduce a simple and scalable method for training Gaussian process (GP) models that exploits cross-validation and nearest neighbor truncation. To accommodate binary and multi-class classification we leverage P\`olya…
ClassificationMulti-class ClassificationregressionVariational InferenceScalable Inference for Gaussian Process Models with Black-Box Likelihoods
We propose a sparse method for scalable automated variational inference (AVI) in a large class of models with Gaussian process (GP) priors, multiple latent functions, multiple outputs and non-linear likelihoods. Our appr…
General ClassificationregressionVariational InferenceScalable Variational Gaussian Process Classification
Gaussian process classification is a popular method with a number of appealing properties. We show how to scale the model within a variational inducing point framework, outperforming the state of the art on benchmark dat…
ClassificationGeneral ClassificationSimplex-to-Euclidean Bijection for Conjugate and Calibrated Multiclass Gaussian Process Classification
We propose a conjugate and calibrated Gaussian process (GP) model for multi-class classification by exploiting the geometry of the probability simplex. Our approach uses Aitchison geometry to map simplex-valued class pro…
Multi-class Classification