Posterior Collapse of a Linear Latent Variable Model

2022-05-09 · ZiHao Wang, Liu Ziyin

This work identifies the existence and cause of a type of posterior collapse that frequently occurs in the Bayesian deep learning practice. For a general linear latent variable model that includes linear variational autoencoders as a special case, we precisely identify the nature of posterior collapse to be the competition between the likelihood and the regularization of the mean due to the prior. Our result suggests that posterior collapse may be related to neural collapse and dimensional collapse and could be a subclass of a general problem of learning for deeper architectures.

📄 PDF Abstract BibTeX arXiv:2205.04009

Code (0)

등록된 구현이 없습니다.

Tasks

model

Understanding Posterior Collapse in Generative Latent Variable Models

2019-03-27 · ICLR Workshop DeepGenStruct 2019 · James Lucas, George Tucker, Roger Grosse, Mohammad Norouzi

Posterior collapse in Variational Autoencoders (VAEs) arises when the variational distribution closely matches the uninformative prior for a subset of latent variables. This paper presents a simple and intuitive explanat…

Variational Inference

Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoders

2023-06-08 · Hien Dang, Tho Tran, Tan Nguyen, Nhat Ho

The posterior collapse phenomenon in variational autoencoder (VAE), where the variational posterior distribution closely matches the prior distribution, can hinder the quality of the learned latent variables. As a conseq…

Decoder

Posterior Collapse as Automatic Spectral Pruning

2026-05-21 · Johannes Hirn arxiv

We show that posterior collapse in $β$-VAEs implements automatic spectral pruning. A latent mode collapses if its contribution to reconstruction is below the cutoff set by $β$. Equilibrium solutions with different $β$ th…

Posterior Collapse and Latent Variable Non-identifiability

2023-01-02 · NeurIPS 2021 12 · Yixin Wang, David M. Blei, John P. Cunningham

Variational autoencoders model high-dimensional data by positing low-dimensional latent variables that are mapped through a flexible distribution parametrized by a neural network. Unfortunately, variational autoencoders …

Attribute Variational Inference

Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse

2019-11-06 · NeurIPS 2019 12 · James Lucas, George Tucker, Roger Grosse, Mohammad Norouzi

Posterior collapse in Variational Autoencoders (VAEs) arises when the variational posterior distribution closely matches the prior for a subset of latent variables. This paper presents a simple and intuitive explanation …