Mechanistic Independence: A Principle for Identifiable Disentangled Representations
Disentangled representations seek to recover latent factors of variation underlying observed data, yet their identifiability is still not fully understood. We introduce a unified framework in which disentanglement is achieved through mechanistic independence, which characterizes latent factors by how they act on observed variables rather than by their latent distribution. This perspective is invariant to changes of the latent density, even when such changes induce statistical dependencies among factors. Within this framework, we propose several related independence criteria -- ranging from support-based and sparsity-based to higher-order conditions -- and show that each yields identifiability of latent subspaces, even under nonlinear, non-invertible mixing. We further establish a hierarchy among these criteria and provide a graph-theoretic characterization of latent subspaces as connected components. Together, these results clarify the conditions under which disentangled representations can be identified without relying on statistical assumptions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Towards Better Understanding of Disentangled Representations via Mutual Information
Most existing works on disentangled representation learning are solely built upon an marginal independence assumption: all factors in disentangled representations should be statistically independent. This assumption is n…
DisentanglementInductive BiasRelationRepresentation LearningIndiSeek learns information-guided disentangled representations
Learning disentangled representations is a fundamental task in multi-modal learning. In modern applications such as single-cell multi-omics, both shared and modality-specific features are critical for characterizing cell…
Representation LearningIndependence Promoted Graph Disentangled Networks
We address the problem of disentangled representation learning with independent latent factors in graph convolutional networks (GCNs). The current methods usually learn node representation by describing its neighborhood …
ClusteringGraph ClassificationGraph ClusteringRepresentation LearningLearning Network Representations with Disentangled Graph Auto-Encoder
The (variational) graph auto-encoder is widely used to learn representations for graph-structured data. However, the formation of real-world graphs is a complicated and heterogeneous process influenced by latent factors.…
DecoderLearning Network RepresentationsDisentangled Representation Learning via Flow Matching
Disentangled representation learning aims to capture the underlying explanatory factors of observed data, enabling a principled understanding of the data-generating process. Recent advances in generative modeling have in…
Representation Learning