Towards an Improved Metric for Evaluating Disentangled Representations
Disentangled representation learning plays a pivotal role in making representations controllable, interpretable and transferable. Despite its significance in the domain, the quest for reliable and consistent quantitative disentanglement metric remains a major challenge. This stems from the utilisation of diverse metrics measuring different properties and the potential bias introduced by their design. Our work undertakes a comprehensive examination of existing popular disentanglement evaluation metrics, comparing them in terms of measuring aspects of disentanglement (viz. Modularity, Compactness, and Explicitness), detecting the factor-code relationship, and describing the degree of disentanglement. We propose a new framework for quantifying disentanglement, introducing a metric entitled \emph{EDI}, that leverages the intuitive concept of \emph{exclusivity} and improved factor-code relationship to minimize ad-hoc decisions. An in-depth analysis reveals that EDI measures essential properties while offering more stability than existing metrics, advocating for its adoption as a standardised approach.
Code (1)
Tasks
DisentanglementRepresentation LearningSimilar Papers 제목 키워드 기반
Enriching Disentanglement: From Logical Definitions to Quantitative Metrics
Disentangling the explanatory factors in complex data is a promising approach for generalizable and data-efficient representation learning. While a variety of quantitative metrics for learning and evaluating disentangled…
DisentanglementRepresentation LearningGeometry of Deep Generative Models for Disentangled Representations
Deep generative models like variational autoencoders approximate the intrinsic geometry of high dimensional data manifolds by learning low-dimensional latent-space variables and an embedding function. The geometric prope…
Representation LearningExplaining Word Embeddings via Disentangled Representation
Disentangled representations have attracted increasing attention recently. However, how to transfer the desired properties of disentanglement to word representations is unclear. In this work, we propose to transform typi…
DisentanglementWord EmbeddingsGM Score: Incorporating inter-class and intra-class generator diversity, discriminability of disentangled representation, and sample fidelity for evaluating GANs
While generative adversarial networks (GAN) are popular for their higher sample quality as opposed to other generative models like the variational autoencoders (VAE) and Boltzmann machines, they suffer from the same diff…
DiversityGauge theory and twins paradox of disentangled representations
Achieving disentangled representations of information is one of the key goals of deep network based machine learning system. Recently there are more discussions on this issue. In this paper, by comparing the geometric st…
BIG-bench Machine Learning