Streaming Variational Inference for Bayesian Nonparametric Mixture Models
In theory, Bayesian nonparametric (BNP) models are well suited to streaming data scenarios due to their ability to adapt model complexity with the observed data. Unfortunately, such benefits have not been fully realized in practice; existing inference algorithms are either not applicable to streaming applications or not extensible to BNP models. For the special case of Dirichlet processes, streaming inference has been considered. However, there is growing interest in more flexible BNP models building on the class of normalized random measures (NRMs). We work within this general framework and present a streaming variational inference algorithm for NRM mixture models. Our algorithm is based on assumed density filtering (ADF), leading straightforwardly to expectation propagation (EP) for large-scale batch inference as well. We demonstrate the efficacy of the algorithm on clustering documents in large, streaming text corpora.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringVariational InferenceSimilar Papers 제목 키워드 기반
Streaming, Distributed Variational Inference for Bayesian Nonparametrics
This paper presents a methodology for creating streaming, distributed inference algorithms for Bayesian nonparametric (BNP) models. In the proposed framework, processing nodes receive a sequence of data minibatches, comp…
Combinatorial OptimizationVariational InferenceTruncation-free Online Variational Inference for Bayesian Nonparametric Models
We present a truncation-free online variational inference algorithm for Bayesian nonparametric models. Unlike traditional (online) variational inference algorithms that require truncations for the model or the variationa…
Topic ModelsVariational InferenceEvaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics
Bayesian models based on the Dirichlet process and other stick-breaking priors have been proposed as core ingredients for clustering, topic modeling, and other unsupervised learning tasks. Prior specification is, however…
SensitivityTruly Nonparametric Online Variational Inference for Hierarchical Dirichlet Processes
Variational methods provide a computationally scalable alternative to Monte Carlo methods for large-scale, Bayesian nonparametric learning. In practice, however, conventional batch and online variational methods quickly…
Continual LearningVariational InferenceStreaming Adaptive Nonparametric Variational Autoencoder
We develop a data driven approach to perform clustering and end-to-end feature learning simultaneously for streaming data that can adaptively detect novel clusters in emerging data. Our approach, Adaptive Nonparametric V…
ClusteringFeature EngineeringVariational Inference