Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing
In statistical learning for real-world large-scale data problems, one must often resort to "streaming" algorithms which operate sequentially on small batches of data. In this work, we present an analysis of the information-theoretic limits of mini-batch inference in the context of generalized linear models and low-rank matrix factorization. In a controlled Bayes-optimal setting, we characterize the optimal performance and phase transitions as a function of mini-batch size. We base part of our results on a detailed analysis of a mini-batch version of the approximate message-passing algorithm (Mini-AMP), which we introduce. Additionally, we show that this theoretical optimality carries over into real-data problems by illustrating that Mini-AMP is competitive with standard streaming algorithms for clustering.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceClusteringSimilar Papers 제목 키워드 기반
Streaming, Distributed Variational Inference for Bayesian Nonparametrics
This paper presents a methodology for creating streaming, distributed inference algorithms for Bayesian nonparametric (BNP) models. In the proposed framework, processing nodes receive a sequence of data minibatches, comp…
Combinatorial OptimizationVariational InferenceStreaming PAC-Bayes Gaussian process regression with a performance guarantee for online decision making
As a powerful Bayesian non-parameterized algorithm, the Gaussian process (GP) has performed a significant role in Bayesian optimization and signal processing. GPs have also advanced online decision-making systems because…
Bayesian OptimizationDecision MakingregressionStreaming Bayesian Inference for Crowdsourced Classification
A key challenge in crowdsourcing is inferring the ground truth from noisy and unreliable data. To do so, existing approaches rely on collecting redundant information from the crowd, and aggregating it with some probabili…
Bayesian InferenceBinary ClassificationClassificationGeneral ClassificationStreaming Variational Inference for Bayesian Nonparametric Mixture Models
In theory, Bayesian nonparametric (BNP) models are well suited to streaming data scenarios due to their ability to adapt model complexity with the observed data. Unfortunately, such benefits have not been fully realized …
ClusteringVariational InferenceCoresets for Scalable Bayesian Logistic Regression
The use of Bayesian methods in large-scale data settings is attractive because of the rich hierarchical models, uncertainty quantification, and prior specification they provide. Standard Bayesian inference algorithms are…
Bayesian InferenceregressionUncertainty Quantification