State Space LSTM Models with Particle MCMC Inference
Long Short-Term Memory (LSTM) is one of the most powerful sequence models. Despite the strong performance, however, it lacks the nice interpretability as in state space models. In this paper, we present a way to combine the best of both worlds by introducing State Space LSTM (SSL) models that generalizes the earlier work \cite{zaheer2017latent} of combining topic models with LSTM. However, unlike \cite{zaheer2017latent}, we do not make any factorization assumptions in our inference algorithm. We present an efficient sampler based on sequential Monte Carlo (SMC) method that draws from the joint posterior directly. Experimental results confirms the superiority and stability of this SMC inference algorithm on a variety of domains.
Code (0)
등록된 구현이 없습니다.
Tasks
State Space ModelsTopic ModelsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Stochastic Gradient MCMC for Nonlinear State Space Models
State space models (SSMs) provide a flexible framework for modeling complex time series via a latent stochastic process. Inference for nonlinear, non-Gaussian SSMs is often tackled with particle methods that do not scale…
Bayesian InferenceState Space ModelsTime SeriesTime Series AnalysisParticle Gibbs with Ancestor Sampling
Particle Markov chain Monte Carlo (PMCMC) is a systematic way of combining the two main tools used for Monte Carlo statistical inference: sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC). We present a nov…
State Space ModelsParticle-MALA and Particle-mGRAD: Gradient-based MCMC methods for high-dimensional state-space models
State-of-the-art methods for Bayesian inference in state-space models are (a) conditional sequential Monte Carlo (CSMC) algorithms; (b) sophisticated 'classical' MCMC algorithms like MALA, or mGRAD from Titsias and Papas…
Bayesian InferenceState Space ModelsParameter elimination in particle Gibbs sampling
Bayesian inference in state-space models is challenging due to high-dimensional state trajectories. A viable approach is particle Markov chain Monte Carlo, combining MCMC and sequential Monte Carlo to form "exact approxi…
Bayesian InferenceEpidemiologyProbabilistic ProgrammingState Space ModelsAugmentation Schemes for Particle MCMC
Particle MCMC involves using a particle filter within an MCMC algorithm. For inference of a model which involves an unobserved stochastic process, the standard implementation uses the particle filter to propose new value…