Factorized Asymptotic Bayesian Inference for Factorial Hidden Markov Models

2015-06-26 · Shaohua Li, Ryohei Fujimaki, Chunyan Miao

Factorial hidden Markov models (FHMMs) are powerful tools of modeling sequential data. Learning FHMMs yields a challenging simultaneous model selection issue, i.e., selecting the number of multiple Markov chains and the dimensionality of each chain. Our main contribution is to address this model selection issue by extending Factorized Asymptotic Bayesian (FAB) inference to FHMMs. First, we offer a better approximation of marginal log-likelihood than the previous FAB inference. Our key idea is to integrate out transition probabilities, yet still apply the Laplace approximation to emission probabilities. Second, we prove that if there are two very similar hidden states in an FHMM, i.e. one is redundant, then FAB will almost surely shrink and eliminate one of them, making the model parsimonious. Experimental results show that FAB for FHMMs significantly outperforms state-of-the-art nonparametric Bayesian iFHMM and Variational FHMM in model selection accuracy, with competitive held-out perplexity.

📄 PDF Abstract BibTeX arXiv:1506.07959

Code (0)

등록된 구현이 없습니다.

Tasks

Bayesian InferenceModel Selection

Factorized Asymptotic Bayesian Inference for Latent Feature Models

2013-12-01 · NeurIPS 2013 12 · Kohei Hayashi, Ryohei Fujimaki

This paper extends factorized asymptotic Bayesian (FAB) inference for latent feature models~(LFMs). FAB inference has not been applicable to models, including LFMs, without a specific condition on the Hesqsian matrix of …

Bayesian Inference Computational Efficiency Model Selection

Scalable Model Selection for Belief Networks

2017-12-01 · NeurIPS 2017 12 · Zhao Song, Yusuke Muraoka, Ryohei Fujimaki, Lawrence Carin

We propose a scalable algorithm for model selection in sigmoid belief networks (SBNs), based on the factorized asymptotic Bayesian (FAB) framework. We derive the corresponding generalized factorized information criterion…

model Model Selection

Hamming Ball Auxiliary Sampling for Factorial Hidden Markov Models

2014-12-01 · NeurIPS 2014 12 · Michalis Titsias Rc Aueb, Christopher Yau

We introduce a novel sampling algorithm for Markov chain Monte Carlo-based Bayesian inference for factorial hidden Markov models. This algorithm is based on an auxiliary variable construction that restricts the model spa…

Bayesian Inference

Beyond similarity assessment: Selecting the optimal model for sequence alignment via the Factorized Asymptotic Bayesian algorithm

2017-05-19 · Taikai Takeda, Michiaki Hamada

Pair Hidden Markov Models (PHMMs) are probabilistic models used for pairwise sequence alignment, a quintessential problem in bioinformatics. PHMMs include three types of hidden states: match, insertion and deletion. Most…

Model Selection

Augmented Ensemble MCMC sampling in Factorial Hidden Markov Models

2017-03-24 · Kaspar Märtens, Michalis K. Titsias, Christopher Yau

Bayesian inference for factorial hidden Markov models is challenging due to the exponentially sized latent variable space. Standard Monte Carlo samplers can have difficulties effectively exploring the posterior landscape…