Syntax-Infused Variational Autoencoder for Text Generation
We present a syntax-infused variational autoencoder (SIVAE), that integrates sentences with their syntactic trees to improve the grammar of generated sentences. Distinct from existing VAE-based text generative models, SIVAE contains two separate latent spaces, for sentences and syntactic trees. The evidence lower bound objective is redesigned correspondingly, by optimizing a joint distribution that accommodates two encoders and two decoders. SIVAE works with long short-term memory architectures to simultaneously generate sentences and syntactic trees. Two versions of SIVAE are proposed: one captures the dependencies between the latent variables through a conditional prior network, and the other treats the latent variables independently such that syntactically-controlled sentence generation can be performed. Experimental results demonstrate the generative superiority of SIVAE on both reconstruction and targeted syntactic evaluations. Finally, we show that the proposed models can be used for unsupervised paraphrasing given different syntactic tree templates.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Text Modeling with Syntax-Aware Variational Autoencoders
Syntactic information contains structures and rules about how text sentences are arranged. Incorporating syntax into text modeling methods can potentially benefit both representation learning and generation. Variational …
Representation LearningAPo-VAE: Text Generation in Hyperbolic Space
Natural language often exhibits inherent hierarchical structure ingrained with complex syntax and semantics. However, most state-of-the-art deep generative models learn embeddings only in Euclidean vector space, without …
Language ModelingLanguage ModellingResponse GenerationText GenerationSyntax-Directed Variational Autoencoder for Structured Data
Deep generative models have been enjoying success in modeling continuous data. However it remains challenging to capture the representations for discrete structures with formal grammars and semantics, e.g., computer prog…
DecoderTranslationvalidSyntax-Infused Transformer and BERT models for Machine Translation and Natural Language Understanding
Attention-based models have shown significant improvement over traditional algorithms in several NLP tasks. The Transformer, for instance, is an illustrative example that generates abstract representations of tokens inpu…
Machine TranslationNatural Language UnderstandingPOSTranslationTowards Visually Explaining Variational Autoencoders
Recent advances in Convolutional Neural Network (CNN) model interpretability have led to impressive progress in visualizing and understanding model predictions. In particular, gradient-based visual attention methods have…
Disentanglement