Sequential Learning for Dirichlet Process Mixtures
Dirichlet process mixture model provides a flexible nonparametric framework for unsupervised learning. Monte Carlo based sampling methods always involve heavy computation efforts; conventional variational inference requires careful design of the variational distribution and the conditional expectation. In this work, we treat the DP mixture itself as the variational proposal, and view the given data as drawn samples of the unknown target distribution. We propose an evidence upper bound (EUBO) to act as the surrogate loss, and fit a DP mixture to the given data by minimizing the EUBO, which is equivalent to minimizing the KL-divergence between the target distribution and the DP mixture. We provide three advantages of the EUBO based DP mixture fitting and show how to build the black-box style sequential learning algorithm. We use the stochastic gradient descent (SGD) algorithm for optimization that leverages on the automatic differentiation tools. Simulation studies are provided to demonstrate the efficiency of our proposed methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Variational InferenceSimilar Papers 제목 키워드 기반
Adaptive Low-Complexity Sequential Inference for Dirichlet Process Mixture Models
We develop a sequential low-complexity inference procedure for Dirichlet process mixtures of Gaussians for online clustering and parameter estimation when the number of clusters are unknown a-priori. We present an easily…
ClusteringOnline Clusteringparameter estimationClustering consistency with Dirichlet process mixtures
Dirichlet process mixtures are flexible non-parametric models, particularly suited to density estimation and probabilistic clustering. In this work we study the posterior distribution induced by Dirichlet process mixture…
ClusteringDensity EstimationUnsupervised Outlier Detection using Random Subspace and Subsampling Ensembles of Dirichlet Process Mixtures
Probabilistic mixture models are recognized as effective tools for unsupervised outlier detection owing to their interpretability and global characteristics. Among these, Dirichlet process mixture models stand out as a s…
Outlier DetectionVariational InferenceDirichlet process mixtures of block $g$ priors for model selection and prediction in linear models
This paper introduces Dirichlet process mixtures of block $g$ priors for model selection and prediction in linear models. These priors are extensions of traditional mixtures of $g$ priors that allow for differential shri…
Model SelectionRetrieval of Experiments with Sequential Dirichlet Process Mixtures in Model Space
We address the problem of retrieving relevant experiments given a query experiment, motivated by the public databases of datasets in molecular biology and other experimental sciences, and the need of scientists to relate…
Retrieval