A Closer Look at Disentangling in $β$-VAE
In many data analysis tasks, it is beneficial to learn representations where each dimension is statistically independent and thus disentangled from the others. If data generating factors are also statistically independent, disentangled representations can be formed by Bayesian inference of latent variables. We examine a generalization of the Variational Autoencoder (VAE), $\beta$-VAE, for learning such representations using variational inference. $\beta$-VAE enforces conditional independence of its bottleneck neurons controlled by its hyperparameter $\beta$. This condition is in general not compatible with the statistical independence of latents. By providing analytical and numerical arguments, we show that this incompatibility leads to a non-monotonic inference performance in $\beta$-VAE with a finite optimal $\beta$.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A closer look at the performance of neural language models on reflexive anaphor licensing
A Closer Look into the Robustness of Neural Dependency Parsers Using Better Adversarial Examples
Disentangling Neural Disjunctive Normal Form Models
Neural Disjunctive Normal Form (DNF) based models are powerful and interpretable approaches to neuro-symbolic learning and have shown promising results in classification and reinforcement learning settings without prior …
Reinforcement LearningNoise or Signal: The Role of Image Backgrounds in Object Recognition
We assess the tendency of state-of-the-art object recognition models to depend on signals from image backgrounds. We create a toolkit for disentangling foreground and background signal on ImageNet images, and find that (…
BIG-bench Machine LearningObject RecognitionA Structural Approach to Growth-at-Risk
We identify the structural impulse responses of quantiles of the outcome variable to a shock. Our estimation strategy explicitly distinguishes treatment from control variables, allowing us to model responses of unconditi…