Low-data? No problem: low-resource, language-agnostic conversational text-to-speech via F0-conditioned data augmentation
The availability of data in expressive styles across languages is limited, and recording sessions are costly and time consuming. To overcome these issues, we demonstrate how to build low-resource, neural text-to-speech (TTS) voices with only 1 hour of conversational speech, when no other conversational data are available in the same language. Assuming the availability of non-expressive speech data in that language, we propose a 3-step technology: 1) we train an F0-conditioned voice conversion (VC) model as data augmentation technique; 2) we train an F0 predictor to control the conversational flavour of the voice-converted synthetic data; 3) we train a TTS system that consumes the augmented data. We prove that our technology enables F0 controllability, is scalable across speakers and languages and is competitive in terms of naturalness over a state-of-the-art baseline model, another augmented method which does not make use of F0 information.
Code (0)
등록된 구현이 없습니다.
Tasks
Data Augmentationtext-to-speechText to SpeechVoice ConversionSimilar Papers 제목 키워드 기반
Zero-shot Cross-lingual Conversational Semantic Role Labeling
While conversational semantic role labeling (CSRL) has shown its usefulness on Chinese conversational tasks, it is still under-explored in non-Chinese languages due to the lack of multilingual CSRL annotations for the pa…
Response GenerationSemantic Role LabelingTranslationPseudo-Labeling for Domain-Agnostic Bangla Automatic Speech Recognition
One of the major challenges for developing automatic speech recognition (ASR) for low-resource languages is the limited access to labeled data with domain-specific variations. In this study, we propose a pseudo-labeling …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionTask-Oriented Dialog Systems for the Senegalese Wolof Language
In recent years, we are seeing considerable interest in conversational agents with the rise of large language models (LLMs). Although they offer considerable advantages, LLMs also present significant risks, such as hallu…
ChatbotHallucinationMachine TranslationDziri Voicebot: An End-to-End Low-Resource Speech-to-Speech Conversational System for Algerian Dialect
Automatic speech and language technologies are still heavily biased toward high-resource languages, limiting their applicability to dialectal and low-resource settings such as Algerian Dialect. This language presents add…
Natural Language UnderstandingText-To-Speech SynthesisIntent ClassificationResponse GenerationZero-shot Cross-lingual Conversational Semantic Role Labeling
While conversational semantic role labeling (CSRL) has shown its usefulness on Chinese conversational tasks, it is still under-explored in non-Chinese languages due to the lack of multilingual CSRL annotations for the pa…
Response GenerationSemantic Role Labeling