A Preliminary Study of Disentanglement With Insights on the Inadequacy of Metrics
Disentangled encoding is an important step towards a better representation learning. However, despite the numerous efforts, there still is no clear winner that captures the independent features of the data in an unsupervised fashion. In this work we empirically evaluate the performance of six unsupervised disentanglement approaches on the mpi3d toy dataset curated and released for the NeurIPS 2019 Disentanglement Challenge. The methods investigated in this work are Beta-VAE, Factor-VAE, DIP-I-VAE, DIP-II-VAE, Info-VAE, and Beta-TCVAE. The capacities of all models were progressively increased throughout the training and the hyper-parameters were kept intact across experiments. The methods were evaluated based on five disentanglement metrics, namely, DCI, Factor-VAE, IRS, MIG, and SAP-Score. Within the limitations of this study, the Beta-TCVAE approach was found to outperform its alternatives with respect to the normalized sum of metrics. However, a qualitative study of the encoded latents reveal that there is not a consistent correlation between the reported metrics and the disentanglement potential of the model.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementRepresentation LearningSimilar Papers 제목 키워드 기반
A Preliminary Study of Disentanglement With Insights on the Inadequacy of Metrics
Disentangled encoding is an important step towards a better representation learning. However, despite the numerous efforts, there still is no clear winner that captures the independent features of the data in an unsuperv…
DisentanglementRepresentation LearningSeeing Your Speech Style: A Novel Zero-Shot Identity-Disentanglement Face-based Voice Conversion
Face-based Voice Conversion (FVC) is a novel task that leverages facial images to generate the target speaker's voice style. Previous work has two shortcomings: (1) suffering from obtaining facial embeddings that are wel…
Contrastive LearningDisentanglementDiversityVoice ConversionMeasuring Disentanglement: A Review of Metrics
Learning to disentangle and represent factors of variation in data is an important problem in AI. While many advances have been made to learn these representations, it is still unclear how to quantify disentanglement. Wh…
DisentanglementA Refutation of Shapley Values for Explainability
Recent work demonstrated the existence of Boolean functions for which Shapley values provide misleading information about the relative importance of features in rule-based explanations. Such misleading information was br…
Bias Discovery within Human Raters: A Case Study of the Jigsaw Dataset
Understanding and quantifying the bias introduced by human annotation of data is a crucial problem for trustworthy supervised learning. Recently, a perspectivist trend has emerged in the NLP community, focusing on the in…