Towards Principled Objectives for Contrastive Disentanglement
Unsupervised learning is an important tool that has received a significant amount of attention for decades. Its goal is unsupervised recovery,' i.e., extracting salient factors/properties from unlabeled data. Because of the challenges in defining salient properties, recently, contrastive disentanglement' has gained popularity to discover the additional variations that are enhanced in one dataset relative to another. %In fact, contrastive disentanglement and unsupervised recovery are often combined in that we seek additional variations that exhibit salient factors/properties.
Existing formulations have devised a variety of losses for this task. However, all present day methods exhibit two major shortcomings: (1) encodings for data that does not exhibit salient factors is not pushed to carry no signal; and (2) introduced losses are often hard to estimate and require additional trainable parameters. We present a new formulation for contrastive disentanglement which avoids both shortcomings by carefully formulating a probabilistic model and by using non-parametric yet easily computable metrics. We show on four challenging datasets that the proposed approach is able to better disentangle salient factors.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementSimilar Papers 제목 키워드 기반
Contrastive-Augmented Flow Matching for Style-Content Disentanglement
Learning representations that separate content and style is crucial for controllable generation and compositional generalization. However, diffusion and flow-based models trained primarily with generative objectives ofte…
An Empirical Study on Disentanglement of Negative-free Contrastive Learning
Negative-free contrastive learning methods have attracted a lot of attention with simplicity and impressive performances for large-scale pretraining. However, its disentanglement property remains unexplored. In this pape…
Contrastive LearningDisentanglementRepresentation LearningA Principled Framework for Multi-View Contrastive Learning
Contrastive Learning (CL), a leading paradigm in Self-Supervised Learning (SSL), typically relies on pairs of data views generated through augmentation. While multiple augmentations per instance (more than two) improve g…
Self-Supervised LearningContrastive LearningContrastive Speaker Embedding With Sequential Disentanglement
Contrastive speaker embedding assumes that the contrast between the positive and negative pairs of speech segments is attributed to speaker identity only. However, this assumption is incorrect because speech signals cont…
Contrastive LearningDisentanglementDisentangled Shared Representations Improve Morpho-Transcriptomic Integration
Spatial transcriptomics (ST) enables the simultaneous profiling of gene expression and tissue morphology, creating an opportunity to learn multimodal representations capturing shared morpho-transcriptomic structure. Howe…
Representation Learning