Variational Particle Approximations
Approximate inference in high-dimensional, discrete probabilistic models is a central problem in computational statistics and machine learning. This paper describes discrete particle variational inference (DPVI), a new approach that combines key strengths of Monte Carlo, variational and search-based techniques. DPVI is based on a novel family of particle-based variational approximations that can be fit using simple, fast, deterministic search techniques. Like Monte Carlo, DPVI can handle multiple modes, and yields exact results in a well-defined limit. Like unstructured mean-field, DPVI is based on optimizing a lower bound on the partition function; when this quantity is not of intrinsic interest, it facilitates convergence assessment and debugging. Like both Monte Carlo and combinatorial search, DPVI can take advantage of factorization, sequential structure, and custom search operators. This paper defines DPVI particle-based approximation family and partition function lower bounds, along with the sequential DPVI and local DPVI algorithm templates for optimizing them. DPVI is illustrated and evaluated via experiments on lattice Markov Random Fields, nonparametric Bayesian mixtures and block-models, and parametric as well as non-parametric hidden Markov models. Results include applications to real-world spike-sorting and relational modeling problems, and show that DPVI can offer appealing time/accuracy trade-offs as compared to multiple alternatives.
Code (0)
등록된 구현이 없습니다.
Tasks
Spike SortingVariational InferenceSimilar Papers 제목 키워드 기반
Understanding and Accelerating Particle-Based Variational Inference
Particle-based variational inference methods (ParVIs) have gained attention in the Bayesian inference literature, for their capacity to yield flexible and accurate approximations. We explore ParVIs from the perspective o…
Bayesian InferenceVariational InferenceDPVI: A Dynamic-Weight Particle-Based Variational Inference Framework
The recently developed Particle-based Variational Inference (ParVI) methods drive the empirical distribution of a set of \emph{fixed-weight} particles towards a given target distribution $\pi$ by iteratively updating par…
Variational InferenceGaussian Density Parametrization Flow: Particle and Stochastic Approaches
Bayesian inference is intractable for most practical problems and requires approximation schemes with several trade-offs. Variational inference provides one of such approximations which, while powerful, has thus far seen…
Bayesian InferenceVariational InferenceProvably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation
Stein Variational Gradient Descent (SVGD) is a popular variational inference algorithm which simulates an interacting particle system to approximately sample from a target distribution, with impressive empirical performa…
Variational InferenceKernel embedded nonlinear observational mappings in the variational mapping particle filter
Recently, some works have suggested methods to combine variational probabilistic inference with Monte Carlo sampling. One promising approach is via local optimal transport. In this approach, a gradient steepest descent m…