Diversifying Neural Text Generation with Part-of-Speech Guided Softmax and Sampling
Neural text generation models are likely to suffer from the low-diversity problem. Various decoding strategies and training-based methods have been proposed to promote diversity only by exploiting contextual features, but rarely do they consider incorporating syntactic structure clues. In this work, we propose using linguistic annotation, i.e., part-of-speech (POS), to guide the text generation. In detail, we introduce POS Guided Softmax to explicitly model two posterior probabilities: (i) next-POS, and (ii) next-token from the vocabulary of the target POS. A POS Guided Sampling strategy is further proposed to address the low-diversity problem by enriching the diversity of POS. Extensive experiments and human evaluations demonstrate that, compared with existing state-of-the-art methods, our POS Guided Softmax and Sampling (POSG) can generate more diverse text while maintaining comparable quality.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityPOSText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Diversifying Neural Text Generation with Part-of-Speech Guided Softmax and Sampling
Neural text generation models are likely to suffer from the low-diversity problem. Various decoding strategies and training-based methods have been proposed to promote diversity only by exploiting contextual features, bu…
DiversityPOSText GenerationDPP-TTS: Diversifying prosodic features of speech via determinantal point processes
With the rapid advancement in deep generative models, recent neural Text-To-Speech(TTS) models have succeeded in synthesizing human-like speech. There have been some efforts to generate speech with various prosody beyond…
DiversityPoint Processestext-to-speechText to SpeechDiversifying Question Generation over Knowledge Base via External Natural Questions
Previous methods on knowledge base question generation (KBQG) primarily focus on enhancing the quality of a single generated question. Recognizing the remarkable paraphrasing ability of humans, we contend that diverse te…
DiversityNatural QuestionsQuestion AnsweringQuestion Generation+1LivelySpeaker: Towards Semantic-Aware Co-Speech Gesture Generation
Gestures are non-verbal but important behaviors accompanying people's speech. While previous methods are able to generate speech rhythm-synchronized gestures, the semantic context of the speech is generally lacking in th…
Gesture GenerationRhythmVoicebox: Text-Guided Multilingual Universal Speech Generation at Scale
Large-scale generative models such as GPT and DALL-E have revolutionized the research community. These models not only generate high fidelity outputs, but are also generalists which can solve tasks not explicitly taught.…
In-Context LearningSpeech Synthesistext-to-speechText to Speech+1