Topic-Guided Variational Auto-Encoder for Text Generation
We propose a topic-guided variational auto-encoder (TGVAE) model for text generation. Distinct from existing variational auto-encoder (VAE) based approaches, which assume a simple Gaussian prior for latent code, our model specifies the prior as a Gaussian mixture model (GMM) parametrized by a neural topic module. Each mixture component corresponds to a latent topic, which provides a guidance to generate sentences under the topic. The neural topic module and the VAE-based neural sequence module in our model are learned jointly. In particular, a sequence of invertible Householder transformations is applied to endow the approximate posterior of the latent code with high flexibility during the model inference. Experimental results show that our TGVAE outperforms its competitors on both unconditional and conditional text generation, which can also generate semantically-meaningful sentences with various topics.
Code (0)
등록된 구현이 없습니다.
Tasks
Conditional Text GenerationText GenerationSimilar Papers 제목 키워드 기반
Topic-Guided Variational Autoencoders for Text Generation
We propose a topic-guided variational autoencoder (TGVAE) model for text generation. Distinct from existing variational autoencoder (VAE) based approaches, which assume a simple Gaussian prior for the latent code, our mo…
Conditional Text GenerationText GenerationDirichlet Variational Autoencoder for Text Modeling
We introduce an improved variational autoencoder (VAE) for text modeling with topic information explicitly modeled as a Dirichlet latent variable. By providing the proposed model topic awareness, it is more superior at r…
Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided Document Generation
This paper introduces a novel approach for topic modeling utilizing latent codebooks from Vector-Quantized Variational Auto-Encoder~(VQ-VAE), discretely encapsulating the rich information of the pre-trained embeddings su…
Image GenerationLanguage ModelingLanguage ModellingVariational Gaussian Topic Model with Invertible Neural Projections
Neural topic models have triggered a surge of interest in extracting topics from text automatically since they avoid the sophisticated derivations in conventional topic models. However, scarce neural topic models incorpo…
DecoderTopic ModelsWord EmbeddingsInductive Topic Variational Graph Auto-Encoder for Text Classification
Graph convolutional networks (GCNs) have been applied recently to text classification and produced an excellent performance. However, existing GCN-based methods do not assume an explicit latent semantic structure of docu…
ClassificationInductive LearningRepresentation LearningSemi-Supervised Text Classification+2