Generating Sentences from Disentangled Syntactic and Semantic Spaces
Variational auto-encoders (VAEs) are widely used in natural language generation due to the regularization of the latent space. However, generating sentences from the continuous latent space does not explicitly model the syntactic information. In this paper, we propose to generate sentences from disentangled syntactic and semantic spaces. Our proposed method explicitly models syntactic information in the VAE's latent space by using the linearized tree sequence, leading to better performance of language generation. Additionally, the advantage of sampling in the disentangled syntactic and semantic latent spaces enables us to perform novel applications, such as the unsupervised paraphrase generation and syntax-transfer generation. Experimental results show that our proposed model achieves similar or better performance in various tasks, compared with state-of-the-art related work.
Code (1)
Tasks
Paraphrase GenerationText GenerationSimilar Papers 제목 키워드 기반
A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations
We propose a generative model for a sentence that uses two latent variables, with one intended to represent the syntax of the sentence and the other to represent its semantics. We show we can achieve better disentangleme…
DisentanglementSemantic SimilaritySemantic Textual SimilaritySentenceLearning Disentangled Representations for Natural Language Definitions
Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. Currently, most disentanglement method…
DisentanglementSentenceLearning Disentangled Representations in Natural Language Definitions with Semantic Role Labeling Supervision
Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. However, most disentanglement methods …
DisentanglementSemantic Role LabelingSentenceDeep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT
We investigate how Multilingual BERT (mBERT) encodes grammar by examining how the high-order grammatical feature of morphosyntactic alignment (how different languages define what counts as a "subject") is manifested acro…
SentenceGraph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders
The injection of syntactic information in Variational AutoEncoders (VAEs) has been shown to result in an overall improvement of performances and generalisation. An effective strategy to achieve such a goal is to separate…
Language ModellingMulti-Task Learning