Adversarial Mutual Information for Text Generation
Recent advances in maximizing mutual information (MI) between the source and target have demonstrated its effectiveness in text generation. However, previous works paid little attention to modeling the backward network of MI (i.e., dependency from the target to the source), which is crucial to the tightness of the variational information maximization lower bound. In this paper, we propose Adversarial Mutual Information (AMI): a text generation framework which is formed as a novel saddle point (min-max) optimization aiming to identify joint interactions between the source and target. Within this framework, the forward and backward networks are able to iteratively promote or demote each other's generated instances by comparing the real and synthetic data distributions. We also develop a latent noise sampling strategy that leverages random variations at the high-level semantic space to enhance the long term dependency in the generation process. Extensive experiments based on different text generation tasks demonstrate that the proposed AMI framework can significantly outperform several strong baselines, and we also show that AMI has potential to lead to a tighter lower bound of maximum mutual information for the variational information maximization problem.
Code (1)
Tasks
Text GenerationSimilar Papers 제목 키워드 기반
On The Utility of Conditional Generation Based Mutual Information for Characterizing Adversarial Subspaces
Recent studies have found that deep learning systems are vulnerable to adversarial examples; e.g., visually unrecognizable adversarial images can easily be crafted to result in misclassification. The robustness of neural…
Exploring Adversarial Examples and Adversarial Robustness of Convolutional Neural Networks by Mutual Information
A counter-intuitive property of convolutional neural networks (CNNs) is their inherent susceptibility to adversarial examples, which severely hinders the application of CNNs in security-critical fields. Adversarial examp…
Adversarial RobustnessMutual Information Maximizing Quantum Generative Adversarial Network and Its Applications in Finance
One of the most promising applications in the era of NISQ (Noisy Intermediate-Scale Quantum) computing is quantum machine learning. Quantum machine learning offers significant quantum advantages over classical machine le…
Generative Adversarial NetworkImage GenerationQuantum Machine LearningDisentangled Text Representation Learning with Information-Theoretic Perspective for Adversarial Robustness
Adversarial vulnerability remains a major obstacle to constructing reliable NLP systems. When imperceptible perturbations are added to raw input text, the performance of a deep learning model may drop dramatically under …
Adversarial RobustnessRepresentation Learningtext-classificationText ClassificationLearning to Disentangle Textual Representations and Attributes via Mutual Information
Learning disentangled representations of textual data is essential for many natural language tasks such as fair classification (\textit{e.g.} building classifiers whose decisions cannot disproportionately hurt or benefi…
AttributeDisentanglementSentenceStyle Transfer