Distribution Matching in Variational Inference
With the increasingly widespread deployment of generative models, there is a mounting need for a deeper understanding of their behaviors and limitations. In this paper, we expose the limitations of Variational Autoencoders (VAEs), which consistently fail to learn marginal distributions in both latent and visible spaces. We show this to be a consequence of learning by matching conditional distributions, and the limitations of explicit model and posterior distributions. It is popular to consider Generative Adversarial Networks (GANs) as a means of overcoming these limitations, leading to hybrids of VAEs and GANs. We perform a large-scale evaluation of several VAE-GAN hybrids and analyze the implications of class probability estimation for learning distributions. While promising, we conclude that at present, VAE-GAN hybrids have limited applicability: they are harder to scale, evaluate, and use for inference compared to VAEs; and they do not improve over the generation quality of GANs.
Code (0)
등록된 구현이 없습니다.
Tasks
Variational InferenceSimilar Papers 제목 키워드 기반
Semi-Implicit Variational Inference via Score Matching
Semi-implicit variational inference (SIVI) greatly enriches the expressiveness of variational families by considering implicit variational distributions defined in a hierarchical manner. However, due to the intractable d…
Bayesian InferenceDenoisingVariational InferenceVariational Inference with Gaussian Score Matching
Variational inference (VI) is a method to approximate the computationally intractable posterior distributions that arise in Bayesian statistics. Typically, VI fits a simple parametric distribution to the target posterior…
Bayesian InferenceVariational InferenceVariational Rectified Flow Matching
We study Variational Rectified Flow Matching, a framework that enhances classic rectified flow matching by modeling multi-modal velocity vector-fields. At inference time, classic rectified flow matching 'moves' samples f…
Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration
Semi-implicit variational inference (SIVI) has been introduced to expand the analytical variational families by defining expressive semi-implicit distributions in a hierarchical manner. However, the single-layer architec…
Bayesian InferenceVariational InferenceAmortized Variational Inference for Simple Hierarchical Models
It is difficult to use subsampling with variational inference in hierarchical models since the number of local latent variables scales with the dataset. Thus, inference in hierarchical models remains a challenge at large…
Variational Inference