Unsupervised Quantized Prosody Representation for Controllable Speech Synthesis
In this paper, we propose a novel prosody disentangle method for prosodic Text-to-Speech (TTS) model, which introduces the vector quantization (VQ) method to the auxiliary prosody encoder to obtain the decomposed prosody representations in an unsupervised manner. Rely on its advantages, the speaking styles, such as pitch, speaking velocity, local pitch variance, etc., are decomposed automatically into the latent quantize vectors. We also investigate the internal mechanism of VQ disentangle process by means of a latent variables counter and find that higher value dimensions usually represent prosody information. Experiments show that our model can control the speaking styles of synthesis results by directly manipulating the latent variables. The objective and subjective evaluations illustrated that our model outperforms the popular models.
Code (0)
등록된 구현이 없습니다.
Tasks
QuantizationSpeech Synthesistext-to-speechText to SpeechSimilar Papers 제목 키워드 기반
Unsupervised word-level prosody tagging for controllable speech synthesis
Although word-level prosody modeling in neural text-to-speech (TTS) has been investigated in recent research for diverse speech synthesis, it is still challenging to control speech synthesis manually without a specific r…
Speech Synthesistext-to-speechText to SpeechApplying the Information Bottleneck Principle to Prosodic Representation Learning
This paper describes a novel design of a neural network-based speech generation model for learning prosodic representation.The problem of representation learning is formulated according to the information bottleneck (IB)…
Representation LearningProsoSpeech: Enhancing Prosody With Quantized Vector Pre-training in Text-to-Speech
Expressive text-to-speech (TTS) has become a hot research topic recently, mainly focusing on modeling prosody in speech. Prosody modeling has several challenges: 1) the extracted pitch used in previous prosody modeling w…
text-to-speechText to SpeechControllable speech synthesis by learning discrete phoneme-level prosodic representations
In this paper, we present a novel method for phoneme-level prosody control of F0 and duration using intuitive discrete labels. We propose an unsupervised prosodic clustering process which is used to discretize phoneme-le…
ClusteringSpeech Synthesistext-to-speechText to SpeechControllable Neural Prosody Synthesis
Speech synthesis has recently seen significant improvements in fidelity, driven by the advent of neural vocoders and neural prosody generators. However, these systems lack intuitive user controls over prosody, making the…
Speech Synthesis