Unpicking Data at the Seams: VAEs, Disentanglement and Independent Components
Disentanglement, or identifying salient statistically independent factors of the data, is of interest in many areas of machine learning and statistics, such as synthetic data generation with controlled properties, robust classification of features, parsimonious encoding, and improving our understanding of the generative process underlying the data. Disentanglement is observed in several generative paradigms, including Variational Autoencoders (VAEs), Generative Adversarial Networks and diffusion models. Particular progress has recently been made in understanding disentanglement in VAEs, where the choice of diagonal posterior covariance matrices is proposed to promote mutual orthogonality between columns of the decoder's Jacobian. We continue this thread to show how such linear independence translates to statistical independence, completing the chain in understanding how the VAE's objective identifies independent components of, or disentangles, the data.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementRobust classificationSynthetic Data GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Local Disentanglement in Variational Auto-Encoders Using Jacobian $L_1$ Regularization
There have been many recent advances in representation learning; however, unsupervised representation learning can still struggle with model identification issues related to rotations of the latent space. Variational Aut…
DisentanglementRepresentation LearningOut of Distribution Reasoning by Weakly-Supervised Disentangled Logic Variational Autoencoder
Out-of-distribution (OOD) detection, i.e., finding test samples derived from a different distribution than the training set, as well as reasoning about such samples (OOD reasoning), are necessary to ensure the safety of …
DisentanglementOut of Distribution (OOD) DetectionDisentangling Granularity: An Implicit Inductive Bias in Factorized VAEs
Despite the success in learning semantically meaningful, unsupervised disentangled representations, variational autoencoders (VAEs) and their variants face a fundamental theoretical challenge: substantial evidence indica…
DisentanglementInductive BiasCFASL: Composite Factor-Aligned Symmetry Learning for Disentanglement in Variational AutoEncoder
Symmetries of input and latent vectors have provided valuable insights for disentanglement learning in VAEs. However, only a few works were proposed as an unsupervised method, and even these works require known factor in…
DecoderDisentanglementInductive BiasDisentanglement by means of action-induced representations
Learning interpretable representations with variational autoencoders (VAEs) is a major goal of representation learning. The main challenge lies in obtaining disentangled representations, where each latent dimension corre…
Representation Learning