Transfer Learning Framework for Low-Resource Text-to-Speech using a Large-Scale Unlabeled Speech Corpus
Training a text-to-speech (TTS) model requires a large scale text labeled speech corpus, which is troublesome to collect. In this paper, we propose a transfer learning framework for TTS that utilizes a large amount of unlabeled speech dataset for pre-training. By leveraging wav2vec2.0 representation, unlabeled speech can highly improve performance, especially in the lack of labeled speech. We also extend the proposed method to zero-shot multi-speaker TTS (ZS-TTS). The experimental results verify the effectiveness of the proposed method in terms of naturalness, intelligibility, and speaker generalization. We highlight that the single speaker TTS model fine-tuned on the only 10 minutes of labeled dataset outperforms the other baselines, and the ZS-TTS model fine-tuned on the only 30 minutes of single speaker dataset can generate the voice of the arbitrary speaker, by pre-training on unlabeled multi-speaker speech corpus.
Code (0)
등록된 구현이 없습니다.
Tasks
text-to-speechText to SpeechTransfer LearningZero-Shot Multi-Speaker TTSSimilar Papers 제목 키워드 기반
Speech-to-Text Translation with Phoneme-Augmented CoT: Enhancing Cross-Lingual Transfer in Low-Resource Scenarios
We propose a Speech-to-Text Translation (S2TT) approach that integrates phoneme representations into a Chain-of-Thought (CoT) framework to improve translation in low-resource and zero-resource settings. By introducing ph…
Cross-Lingual TransferPhoneme RecognitionSpeech-to-TextSpeech-to-Text Translation+1Cross-lingual Transfer for Speech Processing using Acoustic Language Similarity
Speech processing systems currently do not support the vast majority of languages, in part due to the lack of data in low-resource languages. Cross-lingual transfer offers a compelling way to help bridge this digital div…
Cross-Lingual Transferspeech-recognitionSpeech RecognitionSpeech SynthesisSpeech Synthesis for Low Resource Languages using Transliteration Enabled Transfer Learning
In the area of Human Computer Interaction (HCI), Text To Speech (TTS) synthesis has received a significant boost in recent years, especially with the development of various deep learning techniques capable of generating …
speech-recognitionSpeech RecognitionSpeech Synthesistext-to-speech+3Transfer Learning for Less-Resourced Semitic Languages Speech Recognition: the Case of Amharic
While building automatic speech recognition (ASR) requires a large amount of speech and text data, the problem gets worse for less-resourced languages. In this paper, we investigate a model adaptation method, namely tran…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition+1End-to-end Text-to-speech for Low-resource Languages by Cross-Lingual Transfer Learning
End-to-end text-to-speech (TTS) has shown great success on large quantities of paired text plus speech data. However, laborious data collection remains difficult for at least 95% of the languages over the world, which hi…
Cross-Lingual Transfertext-to-speechText to SpeechTransfer Learning