Resetting a fixed broken ELBO
Variational autoencoders (VAEs) are one class of generative probabilistic latent-variable models designed for inference based on known data. They balance reconstruction and regularizer terms. A variational approximation produces an evidence lower bound (ELBO). Multiplying the regularizer term by beta provides a beta-VAE/ELBO, improving disentanglement of the latent space. However, any beta value different than unity violates the laws of conditional probability. To provide a similarly-parameterized VAE, we develop a Renyi (versus Shannon) entropy VAE, and a variational approximation RELBO that introduces a similar parameter. The Renyi VAE has an additional Renyi regularizer-like term with a conditional distribution that is not learned. The term is evaluated essentially analytically using a Singular Value Decomposition method.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementUnitySimilar Papers 제목 키워드 기반
Stochastic Resetting Accelerates Policy Convergence in Reinforcement Learning
Stochastic resetting, where a dynamical process is intermittently returned to a fixed reference state, has emerged as a powerful mechanism for optimizing first-passage properties. Existing theory largely treats static, n…
Reinforcement LearningContinuous ControlFixing a Broken ELBO
Recent work in unsupervised representation learning has focused on learning deep directed latent-variable models. Fitting these models by maximizing the marginal likelihood or evidence is typically intractable, thus a co…
Representation LearningRidge Regression from Poisson Resetting: A Renewal Perspective on Spectral Regularization
We connect stochastic resetting from non-equilibrium statistical physics with ridge regularization in statistical learning. For linear gradient flow, resetting to the origin at rate $r$ produces stationary mean $(X^\top …
Three Variations on Variational Autoencoders
Variational autoencoders (VAEs) are one class of generative probabilistic latent-variable models designed for inference based on known data. We develop three variations on VAEs by introducing a second parameterized encod…
DecoderScalable Resetting Algorithms for Synchronization of Pulse-Coupled Oscillators over Rooted Directed Graphs
We study the problem of robust global synchronization of pulse-coupled oscillators (PCOs) over directed graphs. It is known that when the digraphs are strongly connected, global synchronization can be achieved by using a…