Non-conjugate Variational Message Passing for Multinomial and Binary Regression

2011-12-01 · NeurIPS 2011 12 · David A. Knowles, Tom Minka

Variational Message Passing (VMP) is an algorithmic implementation of the Variational Bayes (VB) method which applies only in the special case of conjugate exponential family models. We propose an extension to VMP, which we refer to as Non-conjugate Variational Message Passing (NCVMP) which aims to alleviate this restriction while maintaining modularity, allowing choice in how expectations are calculated, and integrating into an existing message-passing framework: Infer.NET. We demonstrate NCVMP on logistic binary and multinomial regression. In the multinomial case we introduce a novel variational bound for the softmax factor which is tighter than other commonly used bounds whilst maintaining computational tractability.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

regression

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Bayesian Estimation of Mixed Multinomial Logit Models: Advances and Simulation-Based Evaluations

2019-04-07 · Prateek Bansal, Rico Krueger, Michel Bierlaire, Ricardo A. Daziano 외

Variational Bayes (VB) methods have emerged as a fast and computationally-efficient alternative to Markov chain Monte Carlo (MCMC) methods for scalable Bayesian estimation of mixed multinomial logit (MMNL) models. It has…

SimLDA: A tool for topic model evaluation

2022-08-19 · Rebecca M. C. Taylor, Johan A. du Preez

Variational Bayes (VB) applied to latent Dirichlet allocation (LDA) has become the most popular algorithm for aspect modeling. While sufficiently successful in text topic extraction from large corpora, VB is less success…

model

ALBU: An approximate Loopy Belief message passing algorithm for LDA to improve performance on small data sets

2021-10-01 · Rebecca M. C. Taylor, Johan A. du Preez

BayesPy: Variational Bayesian Inference in Python

2014-10-03 · Jaakko Luttinen

BayesPy is an open-source Python software package for performing variational Bayesian inference. It is based on the variational message passing framework and supports conjugate exponential family models. By removing the …

Bayesian Inference Variational Inference

Information Geometry of Message Passing

2026-08-16 · Mykola Lukashchuk, Kyrylo Yemets, Alex Ledbetter, İsmail Şenöz arxiv

We show that the natural-gradient stationary condition of variational inference has an edge-local form on a Forney-style factor graph. We start from the Bethe free energy and constrain a selected edge marginal to an expo…