Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression
In order to scale standard Gaussian process (GP) regression to large-scale datasets, aggregation models employ factorized training process and then combine predictions from distributed experts. The state-of-the-art aggregation models, however, either provide inconsistent predictions or require time-consuming aggregation process. We first prove the inconsistency of typical aggregations using disjoint or random data partition, and then present a consistent yet efficient aggregation model for large-scale GP. The proposed model inherits the advantages of aggregations, e.g., closed-form inference and aggregation, parallelization and distributed computing. Furthermore, theoretical and empirical analyses reveal that the new aggregation model performs better due to the consistent predictions that converge to the true underlying function when the training size approaches infinity.
Code (1)
Tasks
Distributed ComputingregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Concentration of the matrix-valued minimum mean-square error in optimal Bayesian inference
We consider Bayesian inference of signals with vector-valued entries. Extending concentration techniques from the mathematical physics of spin glasses, we show that the matrix-valued minimum mean-square error concentrate…
Bayesian InferencevalidA Bayesian Committee Machine Potential for Oxygen-containing Organic Compounds
Understanding the pivotal role of oxygen-containing organic compounds in serving as an energy source for living organisms and contributing to protein formation is crucial in the field of biochemistry. This study addresse…
BenchmarkingPositionIntegrating Bayesian and Discriminative Sparse Kernel Machines for Multi-class Active Learning
We propose a novel active learning (AL) model that integrates Bayesian and discriminative kernel machines for fast and accurate multi-class data sampling. By joining a sparse Bayesian model and a maximum margin machine u…
Active LearningHow to choose a Compatible Committee?
Electing a committee of size k from m alternatives (k < m) is an interesting problem under the multi-winner voting rules. However, very few committee selection rules found in the literature consider the coalitional possi…
Distributed Gaussian Processes
To scale Gaussian processes (GPs) to large data sets we introduce the robust Bayesian Committee Machine (rBCM), a practical and scalable product-of-experts model for large-scale distributed GP regression. Unlike state-of…
FormGaussian Processesregression