Multi-type Disentanglement without Adversarial Training
Controlling the style of natural language by disentangling the latent space is an important step towards interpretable machine learning. After the latent space is disentangled, the style of a sentence can be transformed by tuning the style representation without affecting other features of the sentence. Previous works usually use adversarial training to guarantee that disentangled vectors do not affect each other. However, adversarial methods are difficult to train. Especially when there are multiple features (e.g., sentiment, or tense, which we call style types in this paper), each feature requires a separate discriminator for extracting a disentangled style vector corresponding to that feature. In this paper, we propose a unified distribution-controlling method, which provides each specific style value (the value of style types, e.g., positive sentiment, or past tense) with a unique representation. This method contributes a solid theoretical basis to avoid adversarial training in multi-type disentanglement. We also propose multiple loss functions to achieve a style-content disentanglement as well as a disentanglement among multiple style types. In addition, we observe that if two different style types always have some specific style values that occur together in the dataset, they will affect each other when transferring the style values. We call this phenomenon training bias, and we propose a loss function to alleviate such training bias while disentangling multiple types. We conduct experiments on two datasets (Yelp service reviews and Amazon product reviews) to evaluate the style-disentangling effect and the unsupervised style transfer performance on two style types: sentiment and tense. The experimental results show the effectiveness of our model.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementInterpretable Machine LearningSentenceStyle TransferVocal Bursts Type PredictionSimilar Papers 제목 키워드 기반
Decoder-free Robustness Disentanglement without (Additional) Supervision
Adversarial Training (AT) is proposed to alleviate the adversarial vulnerability of machine learning models by extracting only robust features from the input, which, however, inevitably leads to severe accuracy reduction…
BIG-bench Machine LearningDecoderDisentanglementRepresentation LearningMeasuring the Effect of Causal Disentanglement on the Adversarial Robustness of Neural Network Models
Causal Neural Network models have shown high levels of robustness to adversarial attacks as well as an increased capacity for generalisation tasks such as few-shot learning and rare-context classification compared to tra…
Adversarial RobustnessBenchmarkingDisentanglementFew-Shot Learning+2Disentangled Image Generation Through Structured Noise Injection
We explore different design choices for injecting noise into generative adversarial networks (GANs) with the goal of disentangling the latent space. Instead of traditional approaches, we propose feeding multiple noise co…
DisentanglementImage GenerationCoDeGAN: Contrastive Disentanglement for Generative Adversarial Network
Disentanglement, a critical concern in interpretable machine learning, has also garnered significant attention from the computer vision community. Many existing GAN-based class disentanglement (unsupervised) approaches, …
Contrastive LearningDisentanglementGenerative Adversarial NetworkInterpretable Machine LearningNon-uniform Speaker Disentanglement For Depression Detection From Raw Speech Signals
While speech-based depression detection methods that use speaker-identity features, such as speaker embeddings, are popular, they often compromise patient privacy. To address this issue, we propose a speaker disentanglem…
Depression DetectionDisentanglementPrivacy PreservingSpeaker Identification