PAVI: Plate-Amortized Variational Inference
Given some observed data and a probabilistic generative model, Bayesian inference aims at obtaining the distribution of a model's latent parameters that could have yielded the data. This task is challenging for large population studies where thousands of measurements are performed over a cohort of hundreds of subjects, resulting in a massive latent parameter space. This large cardinality renders off-the-shelf Variational Inference (VI) computationally impractical. In this work, we design structured VI families that can efficiently tackle large population studies. To this end, our main idea is to share the parameterization and learning across the different i.i.d. variables in a generative model -symbolized by the model's plates. We name this concept plate amortization, and illustrate the powerful synergies it entitles, resulting in expressive, parsimoniously parameterized and orders of magnitude faster to train large scale hierarchical variational distributions. We illustrate the practical utility of PAVI through a challenging Neuroimaging example featuring a million latent parameters, demonstrating a significant step towards scalable and expressive Variational Inference.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PAVI: Plate-Amortized Variational Inference
Given observed data and a probabilistic generative model, Bayesian inference searches for the distribution of the model's parameters that could have yielded the data. Inference is challenging for large population studies…
Bayesian InferenceVariational InferenceADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models
Frequently, population studies feature pyramidally-organized data represented using Hierarchical Bayesian Models (HBM) enriched with plates. These models can become prohibitively large in settings such as neuroimaging, w…
Density EstimationVariational InferenceMultiple Importance Sampling ELBO and Deep Ensembles of Variational Approximations
In variational inference (VI), the marginal log-likelihood is estimated using the standard evidence lower bound (ELBO), or improved versions as the importance weighted ELBO (IWELBO). We propose the multiple importance sa…
Density EstimationVariational InferenceForward Amortized Inference for Likelihood-Free Variational Marginalization
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-…
General ClassificationVariational InferenceAmortized Inference of Variational Bounds for Learning Noisy-OR
Classical approaches for approximate inference depend on cleverly designed variational distributions and bounds. Modern approaches employ amortized variational inference, which uses a neural network to approximate any po…
Variational Inference