홈 › Papers

Filtering Variational Objectives

2017-05-25 · NeurIPS 2017 12 · Chris J. Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, andriy mnih, Arnaud Doucet, Yee Whye Teh

When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results. Inspired by this, we consider the extension of the ELBO to a family of lower bounds defined by a particle filter's estimator of the marginal likelihood, the filtering variational objectives (FIVOs). FIVOs take the same arguments as the ELBO, but can exploit a model's sequential structure to form tighter bounds. We present results that relate the tightness of FIVO's bound to the variance of the particle filter's estimator by considering the generic case of bounds defined as log-transformed likelihood estimators. Experimentally, we show that training with FIVO results in substantial improvements over training the same model architecture with the ELBO on sequential data.

📄 PDF Abstract BibTeX arXiv:1705.09279

Code (3)

AdrienCorenflos/differentiableFIVO tf

tensorflow/models tf

tensorflow/models/tree/master/research/fivo tf

Monte Carlo Filtering Objectives: A New Family of Variational Objectives to Learn Generative Model and Neural Adaptive Proposal for Time Series

2021-05-20 · Shuangshuang Chen, Sihao Ding, Yiannis Karayiannidis, Mårten Björkman

Learning generative models and inferring latent trajectories have shown to be challenging for time series due to the intractable marginal likelihoods of flexible generative models. It can be addressed by surrogate object…

Time Series Time Series Analysis

Variational Marginal Particle Filters

2021-09-30 · Jinlin Lai, Justin Domke, Daniel Sheldon

Variational inference for state space models (SSMs) is known to be hard in general. Recent works focus on deriving variational objectives for SSMs from unbiased sequential Monte Carlo estimators. We reveal that the margi…

State Space Models Variational Inference

Particle Smoothing Variational Objectives

2019-09-20 · Antonio Khalil Moretti, Zizhao Wang, Luhuan Wu, Iddo Drori 외

A body of recent work has focused on constructing a variational family of filtered distributions using Sequential Monte Carlo (SMC). Inspired by this work, we introduce Particle Smoothing Variational Objectives (SVO), a …

Diversity

Ensemble Kalman Variational Objectives: Nonlinear Latent Trajectory Inference with A Hybrid of Variational Inference and Ensemble Kalman Filter

2020-10-17 · Tsuyoshi Ishizone, Tomoyuki Higuchi, Kazuyuki Nakamura

Variational inference (VI) combined with Bayesian nonlinear filtering produces state-of-the-art results for latent time-series modeling. A body of recent work has focused on sequential Monte Carlo (SMC) and its variants,…

Diversity State Space Models Time Series Time Series Analysis+1

Untangling Lariats: Subgradient Following of Variationally Penalized Objectives

2024-05-07 · Kai-Chia Mo, Shai Shalev-Shwartz, Nisæl Shártov

We describe an apparatus for subgradient-following of the optimum of convex problems with variational penalties. In this setting, we receive a sequence $y_i,\ldots,y_n$ and seek a smooth sequence $x_1,\ldots,x_n$. The sm…

Temporal Sequences