Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech
Whilst recent neural text-to-speech (TTS) approaches produce high-quality speech, they typically require a large amount of recordings from the target speaker. In previous work, a 3-step method was proposed to generate high-quality TTS while greatly reducing the amount of data required for training. However, we have observed a ceiling effect in the level of naturalness achievable for highly expressive voices when using this approach. In this paper, we present a method for building highly expressive TTS voices with as little as 15 minutes of speech data from the target speaker. Compared to the current state-of-the-art approach, our proposed improvements close the gap to recordings by 23.3% for naturalness of speech and by 16.3% for speaker similarity. Further, we match the naturalness and speaker similarity of a Tacotron2-based full-data (~10 hours) model using only 15 minutes of target speaker data, whereas with 30 minutes or more, we significantly outperform it. The following improvements are proposed: 1) changing from an autoregressive, attention-based TTS model to a non-autoregressive model replacing attention with an external duration model and 2) an additional Conditional Generative Adversarial Network (cGAN) based fine-tuning step.
Code (0)
등록된 구현이 없습니다.
Tasks
Generative Adversarial Networktext-to-speechText to SpeechSimilar Papers 제목 키워드 기반
Autoregressive conditional duration modelling of high frequency data
This paper explores the duration dynamics modelling under the Autoregressive Conditional Durations (ACD) framework (Engle and Russell 1998). I test different distributions assumptions for the durations. The empirical res…
Vocal Bursts Intensity PredictionShould you use a probabilistic duration model in TTS? Probably! Especially for spontaneous speech
Converting input symbols to output audio in TTS requires modelling the durations of speech sounds. Leading non-autoregressive (NAR) TTS models treat duration modelling as a regression problem. The same utterance is then …
regressionGenerative Adversarial Training for Text-to-Speech Synthesis Based on Raw Phonetic Input and Explicit Prosody Modelling
We describe an end-to-end speech synthesis system that uses generative adversarial training. We train our Vocoder for raw phoneme-to-audio conversion, using explicit phonetic, pitch and duration modeling. We experiment w…
Speech Synthesistext-to-speechText to SpeechText-To-Speech Synthesis+1TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction
We propose TalkNet, a non-autoregressive convolutional neural model for speech synthesis with explicit pitch and duration prediction. The model consists of three feed-forward convolutional networks. The first network pre…
Speech Synthesistext-to-speechText to SpeechJDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment
We propose Jointly trained Duration Informed Transformer (JDI-T), a feed-forward Transformer with a duration predictor jointly trained without explicit alignments in order to generate an acoustic feature sequence from an…
text-to-speechText to Speech