Deep State Space Models for Unconditional Word Generation
Autoregressive feedback is considered a necessity for successful unconditional text generation using stochastic sequence models. However, such feedback is known to introduce systematic biases into the training process and it obscures a principle of generation: committing to global information and forgetting local nuances. We show that a non-autoregressive deep state space model with a clear separation of global and local uncertainty can be built from only two ingredients: An independent noise source and a deterministic transition function. Recent advances on flow-based variational inference can be used to train an evidence lower-bound without resorting to annealing, auxiliary losses or similar measures. The result is a highly interpretable generative model on par with comparable auto-regressive models on the task of word generation.
Code (0)
등록된 구현이 없습니다.
Tasks
State Space ModelsText GenerationVariational InferenceSimilar Papers 제목 키워드 기반
Return of Unconditional Generation: A Self-supervised Representation Generation Method
Unconditional generation -- the problem of modeling data distribution without relying on human-annotated labels -- is a long-standing and fundamental challenge in generative models, creating a potential of learning from …
Conditional Image GenerationImage GenerationUnconditional Image GenerationAutoregressive Text Generation Beyond Feedback Loops
Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models. However, autoregressive feedback exposes the ev…
SentenceText GenerationUnconditional Image-Text Pair Generation with Multimodal Cross Quantizer
Although deep generative models have gained a lot of attention, most of the existing works are designed for unimodal generation. In this paper, we explore a new method for unconditional image-text pair generation. We des…
multimodal generationQuantizationDisentanglement in a GAN for Unconditional Speech Synthesis
Can we develop a model that can synthesize realistic speech directly from a latent space, without explicit conditioning? Despite several efforts over the last decade, previous adversarial and diffusion-based approaches s…
DisentanglementGenerative Adversarial NetworkImage GenerationSpeaker Verification+3Autoregressive GAN for Semantic Unconditional Head Motion Generation
In this work, we address the task of unconditional head motion generation to animate still human faces in a low-dimensional semantic space from a single reference pose. Different from traditional audio-conditioned talkin…
Motion GenerationTalking Head Generation