Unsupervised Representation Disentanglement of Text: An Evaluation on Synthetic Datasets
To highlight the challenges of achieving representation disentanglement for text domain in an unsupervised setting, in this paper we select a representative set of successfully applied models from the image domain. We evaluate these models on 6 disentanglement metrics, as well as on downstream classification tasks and homotopy. To facilitate the evaluation, we propose two synthetic datasets with known generative factors. Our experiments highlight the existing gap in the text domain and illustrate that certain elements such as representation sparsity (as an inductive bias), or representation coupling with the decoder could impact disentanglement. To the best of our knowledge, our work is the first attempt on the intersection of unsupervised representation disentanglement and text, and provides the experimental framework and datasets for examining future developments in this direction.
Code (1)
Tasks
DecoderDisentanglementInductive BiasSimilar Papers 제목 키워드 기반
Closed-Loop Unsupervised Representation Disentanglement with $β$-VAE Distillation and Diffusion Probabilistic Feedback
Representation disentanglement may help AI fundamentally understand the real world and thus benefit both discrimination and generation tasks. It currently has at least three unresolved core issues: (i) heavy reliance on …
DisentanglementImage Manipulation$β$-CapsNet: Learning Disentangled Representation for CapsNet by Information Bottleneck
We present a framework for learning disentangled representation of CapsNet by information bottleneck constraint that distills information into a compact form and motivates to learn an interpretable factorized capsule. In…
DisentanglementVariational InferenceSemi-Supervised StyleGAN for Disentanglement Learning
Disentanglement learning is crucial for obtaining disentangled representations and controllable generation. Current disentanglement methods face several inherent limitations: difficulty with high-resolution images, prima…
DisentanglementRepresentation LearningUnsupervised Synthetic Image Attribution: Alignment and Disentanglement
As the quality of synthetic images improves, identifying the underlying concepts of model-generated images is becoming increasingly crucial for copyright protection and ensuring model transparency. Existing methods achie…
Self-Supervised LearningImage AttributionLearning Disentangled Representations for Natural Language Definitions
Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. Currently, most disentanglement method…
DisentanglementSentence