Variational Approximation Error in Bayesian Non-negative Matrix Factorization
Non-negative matrix factorization (NMF) is a knowledge discovery method that is used in many fields. Variational inference and Gibbs sampling methods for it are also wellknown. However, the variational approximation error has not been clarified yet, because NMF is not statistically regular and the prior distribution used in variational Bayesian NMF (VBNMF) has zero or divergence points. In this paper, using algebraic geometrical methods, we theoretically analyze the difference in negative log evidence (a.k.a. free energy) between VBNMF and Bayesian NMF, i.e., the Kullback-Leibler divergence between the variational posterior and the true posterior. We derive an upper bound for the learning coefficient (a.k.a. the real log canonical threshold) in Bayesian NMF. By using the upper bound, we find a lower bound for the approximation error, asymptotically. The result quantitatively shows how well the VBNMF algorithm can approximate Bayesian NMF; the lower bound depends on the hyperparameters and the true nonnegative rank. A numerical experiment demonstrates the theoretical result.
Code (1)
Tasks
Variational InferenceSimilar Papers 제목 키워드 기반
1-bit Matrix Completion: PAC-Bayesian Analysis of a Variational Approximation
Due to challenging applications such as collaborative filtering, the matrix completion problem has been widely studied in the past few years. Different approaches rely on different structure assumptions on the matrix in …
Collaborative FilteringLearning TheoryMatrix CompletionA particle-based variational approach to Bayesian Non-negative Matrix Factorization
Bayesian Non-negative Matrix Factorization (NMF) is a promising approach for understanding uncertainty and structure in matrix data. However, a large volume of applied work optimizes traditional non-Bayesian NMF objectiv…
Fast Bayesian Non-Negative Matrix Factorisation and Tri-Factorisation
We present a fast variational Bayesian algorithm for performing non-negative matrix factorisation and tri-factorisation. We show that our approach achieves faster convergence per iteration and timestep (wall-clock) than …
Comparative Study of Inference Methods for Bayesian Nonnegative Matrix Factorisation
In this paper, we study the trade-offs of different inference approaches for Bayesian matrix factorisation methods, which are commonly used for predicting missing values, and for finding patterns in the data. In particul…
Bayesian InferenceMissing ValuesModel SelectionStatistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection
Variational inference has recently emerged as a popular alternative to the classical Markov chain Monte Carlo (MCMC) in large-scale Bayesian inference. The core idea is to trade statistical accuracy for computational eff…
Bayesian InferenceComputational EfficiencyModel SelectionStochastic Optimization+2