Improved Bayesian Logistic Supervised Topic Models with Data Augmentation
Supervised topic models with a logistic likelihood have two issues that potentially limit their practical use: 1) response variables are usually over-weighted by document word counts; and 2) existing variational inference methods make strict mean-field assumptions. We address these issues by: 1) introducing a regularization constant to better balance the two parts based on an optimization formulation of Bayesian inference; and 2) developing a simple Gibbs sampling algorithm by introducing auxiliary Polya-Gamma variables and collapsing out Dirichlet variables. Our augment-and-collapse sampling algorithm has analytical forms of each conditional distribution without making any restricting assumptions and can be easily parallelized. Empirical results demonstrate significant improvements on prediction performance and time efficiency.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceData AugmentationTopic ModelsVariational InferenceSimilar Papers 제목 키워드 기반
Prediction-Constrained Topic Models for Antidepressant Recommendation
Supervisory signals can help topic models discover low-dimensional data representations that are more interpretable for clinical tasks. We propose a framework for training supervised latent Dirichlet allocation that bala…
PredictionTopic ModelsPrediction-Constrained Training for Semi-Supervised Mixture and Topic Models
Supervisory signals have the potential to make low-dimensional data representations, like those learned by mixture and topic models, more interpretable and useful. We propose a framework for training latent variable mode…
PredictionSentiment AnalysisTopic ModelsEvaluating Topic Quality with Posterior Variability
Probabilistic topic models such as latent Dirichlet allocation (LDA) are popularly used with Bayesian inference methods such as Gibbs sampling to learn posterior distributions over topic model parameters. We derive a nov…
Bayesian InferenceTopic ModelsA General Method for Robust Bayesian Modeling
Robust Bayesian models are appealing alternatives to standard models, providing protection from data that contains outliers or other departures from the model assumptions. Historically, robust models were mostly develope…
regressionTopic ModelsDiscriminative Relational Topic Models
Many scientific and engineering fields involve analyzing network data. For document networks, relational topic models (RTMs) provide a probabilistic generative process to describe both the link structure and document con…
Bayesian InferenceData AugmentationTopic Models