Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognition
An unsupervised text-to-speech synthesis (TTS) system learns to generate speech waveforms corresponding to any written sentence in a language by observing: 1) a collection of untranscribed speech waveforms in that language; 2) a collection of texts written in that language without access to any transcribed speech. Developing such a system can significantly improve the availability of speech technology to languages without a large amount of parallel speech and text data. This paper proposes an unsupervised TTS system based on an alignment module that outputs pseudo-text and another synthesis module that uses pseudo-text for training and real text for inference. Our unsupervised system can achieve comparable performance to the supervised system in seven languages with about 10-20 hours of speech each. A careful study on the effect of text units and vocoders has also been conducted to better understand what factors may affect unsupervised TTS performance. The samples generated by our models can be found at https://cactuswiththoughts.github.io/UnsupTTS-Demo, and our code can be found at https://github.com/lwang114/UnsupTTS.
Code (1)
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Sentencespeech-recognitionSpeech RecognitionSpeech Synthesistext-to-speechText to SpeechText-To-Speech SynthesisSimilar Papers 제목 키워드 기반
Improving Accented Speech Recognition using Data Augmentation based on Unsupervised Text-to-Speech Synthesis
This paper investigates the use of unsupervised text-to-speech synthesis (TTS) as a data augmentation method to improve accented speech recognition. TTS systems are trained with a small amount of accented speech training…
Accented Speech RecognitionAutomatic Speech RecognitionAutomatic Speech Recognition (ASR)Data Augmentation+7Simple and Effective Unsupervised Speech Synthesis
We introduce the first unsupervised speech synthesis system based on a simple, yet effective recipe. The framework leverages recent work in unsupervised speech recognition as well as existing neural-based speech synthesi…
speech-recognitionSpeech RecognitionSpeech SynthesisUnsupervised Speech RecognitionDeep Denoising Auto-encoder for Statistical Speech Synthesis
This paper proposes a deep denoising auto-encoder technique to extract better acoustic features for speech synthesis. The technique allows us to automatically extract low-dimensional features from high dimensional spectr…
DenoisingSpeech Synthesistext-to-speechText to SpeechModeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no access to speech data
Human speakers encode information into raw speech which is then decoded by the listeners. This complex relationship between encoding (production) and decoding (perception) is often modeled separately. Here, we test how d…
speech-recognitionSpeech RecognitionSpeech SynthesisDeep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
Generating versatile and appropriate synthetic speech requires control over the output expression separate from the spoken text. Important non-textual speech variation is seldom annotated, in which case output control mu…
Acoustic ModellingDecoderEmotional Speech SynthesisSpeech Synthesis+1