Improved Prosodic Clustering for Multispeaker and Speaker-independent Phoneme-level Prosody Control
This paper presents a method for phoneme-level prosody control of F0 and duration on a multispeaker text-to-speech setup, which is based on prosodic clustering. An autoregressive attention-based model is used, incorporating multispeaker architecture modules in parallel to a prosody encoder. Several improvements over the basic single-speaker method are proposed that increase the prosodic control range and coverage. More specifically we employ data augmentation, F0 normalization, balanced clustering for duration, and speaker-independent prosodic clustering. These modifications enable fine-grained phoneme-level prosody control for all speakers contained in the training set, while maintaining the speaker identity. The model is also fine-tuned to unseen speakers with limited amounts of data and it is shown to maintain its prosody control capabilities, verifying that the speaker-independent prosodic clustering is effective. Experimental results verify that the model maintains high output speech quality and that the proposed method allows efficient prosody control within each speaker's range despite the variability that a multispeaker setting introduces.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringData Augmentationtext-to-speechText to SpeechSimilar Papers 제목 키워드 기반
Controllable speech synthesis by learning discrete phoneme-level prosodic representations
In this paper, we present a novel method for phoneme-level prosody control of F0 and duration using intuitive discrete labels. We propose an unsupervised prosodic clustering process which is used to discretize phoneme-le…
ClusteringSpeech Synthesistext-to-speechText to SpeechExact Prosody Cloning in Zero-Shot Multispeaker Text-to-Speech
The cloning of a speaker's voice using an untranscribed reference sample is one of the great advances of modern neural text-to-speech (TTS) methods. Approaches for mimicking the prosody of a transcribed reference audio h…
text-to-speechText to SpeechSpeaker verification-derived loss and data augmentation for DNN-based multispeaker speech synthesis
Building multispeaker neural network-based text-to-speech synthesis systems commonly relies on the availability of large amounts of high quality recordings from each speaker and conditioning the training process on the s…
Data AugmentationSpeaker VerificationSpeech Synthesistext-to-speech+2Cross-lingual Multispeaker Text-to-Speech under Limited-Data Scenario
Modeling voices for multiple speakers and multiple languages in one text-to-speech system has been a challenge for a long time. This paper presents an extension on Tacotron2 to achieve bilingual multispeaker speech synth…
AttributeSpeech Synthesistext-to-speechText to SpeechCHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings
Following the success of the 1st, 2nd, 3rd, 4th and 5th CHiME challenges we organize the 6th CHiME Speech Separation and Recognition Challenge (CHiME-6). The new challenge revisits the previous CHiME-5 challenge and furt…
speaker-diarizationSpeaker DiarizationSpeech Enhancementspeech-recognition+2