Disentanglement in Difference: Directly Learning Semantically Disentangled Representations by Maximizing Inter-Factor Differences
In this study, Disentanglement in Difference(DiD) is proposed to address the inherent inconsistency between the statistical independence of latent variables and the goal of semantic disentanglement in disentanglement representation learning. Conventional disentanglement methods achieve disentanglement representation by improving statistical independence among latent variables. However, the statistical independence of latent variables does not necessarily imply that they are semantically unrelated, thus, improving statistical independence does not always enhance disentanglement performance. To address the above issue, DiD is proposed to directly learn semantic differences rather than the statistical independence of latent variables. In the DiD, a Difference Encoder is designed to measure the semantic differences; a contrastive loss function is established to facilitate inter-dimensional comparison. Both of them allow the model to directly differentiate and disentangle distinct semantic factors, thereby resolving the inconsistency between statistical independence and semantic disentanglement. Experimental results on the dSprites and 3DShapes datasets demonstrate that the proposed DiD outperforms existing mainstream methods across various disentanglement metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementRepresentation LearningSimilar Papers 제목 키워드 기반
Variational Autoencoder with Disentanglement Priors for Low-Resource Task-Specific Natural Language Generation
In this paper, we propose a variational autoencoder with disentanglement priors, VAE-DPRIOR, for task-specific natural language generation with none or a handful of task-specific labeled examples. In order to tackle comp…
Data AugmentationDisentanglementFew-Shot LearningRepresentation Learning+3Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View
From the intuitive notion of disentanglement, the image variations corresponding to different factors should be distinct from each other, and the disentangled representation should reflect those variations with separate …
Contrastive LearningDisentanglementRepresentation LearningRepresentation Disentanglement for Multi-task Learning with application to Fetal Ultrasound
One of the biggest challenges for deep learning algorithms in medical image analysis is the indiscriminate mixing of image properties, e.g. artifacts and anatomy. These entangled image properties lead to a semantically r…
AnatomyDisentanglementMedical Image AnalysisMulti-Task LearningPhysically Disentangled Representations
State-of-the-art methods in generative representation learning yield semantic disentanglement, but typically do not consider physical scene parameters, such as geometry, albedo, lighting, or camera. We posit that inverse…
AttributeClassificationDisentanglementEmotion Recognition+2Learning Disentangled Representations of Negation and Uncertainty
Negation and uncertainty modeling are long-standing tasks in natural language processing. Linguistic theory postulates that expressions of negation and uncertainty are semantically independent from each other and the con…
DisentanglementNegationRepresentation Learning