Improving Disentangled Text Representation Learning with Information-Theoretic Guidance
Learning disentangled representations of natural language is essential for many NLP tasks, e.g., conditional text generation, style transfer, personalized dialogue systems, etc. Similar problems have been studied extensively for other forms of data, such as images and videos. However, the discrete nature of natural language makes the disentangling of textual representations more challenging (e.g., the manipulation over the data space cannot be easily achieved). Inspired by information theory, we propose a novel method that effectively manifests disentangled representations of text, without any supervision on semantics. A new mutual information upper bound is derived and leveraged to measure dependence between style and content. By minimizing this upper bound, the proposed method induces style and content embeddings into two independent low-dimensional spaces. Experiments on both conditional text generation and text-style transfer demonstrate the high quality of our disentangled representation in terms of content and style preservation.
Code (0)
등록된 구현이 없습니다.
Tasks
Conditional Text GenerationRepresentation LearningStyle TransferText GenerationText Style TransferSimilar Papers 제목 키워드 기반
Disentangled Text Representation Learning with Information-Theoretic Perspective for Adversarial Robustness
Adversarial vulnerability remains a major obstacle to constructing reliable NLP systems. When imperceptible perturbations are added to raw input text, the performance of a deep learning model may drop dramatically under …
Adversarial RobustnessRepresentation Learningtext-classificationText ClassificationDisentangled Representation Learning with Transmitted Information Bottleneck
Encoding only the task-related information from the raw data, \ie, disentangled representation learning, can greatly contribute to the robustness and generalizability of models. Although significant advances have been ma…
DisentanglementRepresentation LearningVariational InferenceComparison Reveals Commonality: Customized Image Generation through Contrastive Inversion
The recent demand for customized image generation raises a need for techniques that effectively extract the common concept from small sets of images. Existing methods typically rely on additional guidance, such as text p…
Contrastive LearningImage GenerationImproving Zero-shot Voice Style Transfer via Disentangled Representation Learning
Voice style transfer, also called voice conversion, seeks to modify one speaker's voice to generate speech as if it came from another (target) speaker. Previous works have made progress on voice conversion with parallel …
DecoderRepresentation LearningStyle TransferVoice ConversionDisentangled Representations using Trained Models
We propose a novel method to learn disentangled representations. The ability to compute a disentangled representation is useful for many tasks because it contains information about samples from a dataset in an interpreta…