Controlled Text Generation Using Dictionary Prior in Variational Autoencoders
While variational autoencoders (VAEs) have been widely applied in text generation tasks, they are troubled by two challenges: insufficient representation capacity and poor controllability. The former results from the posterior collapse and restrictive assumption, which impede better representation learning. The latter arises as continuous latent variables in traditional formulations hinder VAEs from interpretability and controllability. In this paper, we propose Dictionary Prior (DPrior), a new data-driven prior that enjoys the merits of expressivity and controllability. To facilitate controlled text generation with DPrior, we propose to employ contrastive learning to separate the latent space into several parts. Extensive experiments on both language modeling and controlled text generation demonstrate the effectiveness of the proposed approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningLanguage ModelingLanguage ModellingRepresentation LearningText GenerationSimilar Papers 제목 키워드 기반
Controllable Paraphrase Generation with a Syntactic Exemplar
Prior work on controllable text generation usually assumes that the controlled attribute can take on one of a small set of values known a priori. In this work, we propose a novel task, where the syntax of a generated sen…
AttributeParaphrase GenerationRepresentation LearningSentence+1Variational Depth Superresolution Using Example-Based Edge Representations
In this paper we propose a novel method for depth image superresolution which combines recent advances in example based upsampling with variational superresolution based on a known blur kernel. Most traditional depth sup…
A Sparsity-promoting Dictionary Model for Variational Autoencoders
Structuring the latent space in probabilistic deep generative models, e.g., variational autoencoders (VAEs), is important to yield more expressive models and interpretable representations, and to avoid overfitting. One w…
modelVariational InferenceFeedback-Controlled Sequential Lasso Screening
One way to solve lasso problems when the dictionary does not fit into available memory is to first screen the dictionary to remove unneeded features. Prior research has shown that sequential screening methods offer the g…
Model SelectionSparse Bayesian Dictionary Learning with a Gaussian Hierarchical Model
We consider a dictionary learning problem whose objective is to design a dictionary such that the signals admits a sparse or an approximate sparse representation over the learned dictionary. Such a problem finds a variet…
Bayesian InferenceDenoisingDictionary LearningImage Denoising