ED-VAE: Entropy Decomposition of ELBO in Variational Autoencoders
Traditional Variational Autoencoders (VAEs) are constrained by the limitations of the Evidence Lower Bound (ELBO) formulation, particularly when utilizing simplistic, non-analytic, or unknown prior distributions. These limitations inhibit the VAE's ability to generate high-quality samples and provide clear, interpretable latent representations. This work introduces the Entropy Decomposed Variational Autoencoder (ED-VAE), a novel re-formulation of the ELBO that explicitly includes entropy and cross-entropy components. This reformulation significantly enhances model flexibility, allowing for the integration of complex and non-standard priors. By providing more detailed control over the encoding and regularization of latent spaces, ED-VAE not only improves interpretability but also effectively captures the complex interactions between latent variables and observed data, thus leading to better generative performance.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Resetting a fixed broken ELBO
Variational autoencoders (VAEs) are one class of generative probabilistic latent-variable models designed for inference based on known data. They balance reconstruction and regularizer terms. A variational approximation …
DisentanglementUnityThe ELBO of Variational Autoencoders Converges to a Sum of Three Entropies
The central objective function of a variational autoencoder (VAE) is its variational lower bound (the ELBO). Here we show that for standard (i.e., Gaussian) VAEs the ELBO converges to a value given by the sum of three en…
On the Convergence of the ELBO to Entropy Sums
The variational lower bound (a.k.a. ELBO or free energy) is the central objective for many established as well as for many novel algorithms for unsupervised learning. Such algorithms usually increase the bound until para…
Learning variational autoencoders via MCMC speed measures
Variational autoencoders (VAEs) are popular likelihood-based generative models which can be efficiently trained by maximizing an Evidence Lower Bound (ELBO). There has been much progress in improving the expressiveness o…
Unbiased Implicit Variational Inference
We develop unbiased implicit variational inference (UIVI), a method that expands the applicability of variational inference by defining an expressive variational family. UIVI considers an implicit variational distributio…
regressionVariational Inference