Learning pronunciation from a foreign language in speech synthesis networks
Although there are more than 65,000 languages in the world, the pronunciations of many phonemes sound similar across the languages. When people learn a foreign language, their pronunciation often reflect their native language's characteristics. That motivates us to investigate how the speech synthesis network learns the pronunciation when multi-lingual dataset is given. In this study, we train the speech synthesis network bilingually in English and Korean, and analyze how the network learns the relations of phoneme pronunciation between the languages. Our experimental result shows that the learned phoneme embedding vectors are located closer if their pronunciations are similar across the languages. Based on the result, we also show that it is possible to train networks that synthesize English speaker's Korean speech and vice versa. In another experiment, we train the network with limited amount of English dataset and large Korean dataset, and analyze the required amount of dataset to train a resource-poor language with the help of resource-rich languages.
Code (1)
Tasks
Speech SynthesisSimilar Papers 제목 키워드 기반
Learning pronunciation from a foreign language in speech synthesis networks
Although there are more than 6,500 languages in the world, the pronunciations of many phonemes sound similar across the languages. When people learn a foreign language, their pronunciation often reflects their native lan…
Speech SynthesisImproving Cross-lingual Speech Synthesis with Triplet Training Scheme
Recent advances in cross-lingual text-to-speech (TTS) made it possible to synthesize speech in a language foreign to a monolingual speaker. However, there is still a large gap between the pronunciation of generated cross…
Speech Synthesistext-to-speechText to SpeechTripletExploring Retraining-Free Speech Recognition for Intra-sentential Code-Switching
In this paper, we present our initial efforts for building a code-switching (CS) speech recognition system leveraging existing acoustic models (AMs) and language models (LMs), i.e., no training required, and specifically…
Language ModelingLanguage Modellingspeech-recognitionSpeech RecognitionPronunciation Modeling of Foreign Words for Mandarin ASR by Considering the Effect of Language Transfer
One of the challenges in automatic speech recognition is foreign words recognition. It is observed that a speaker's pronunciation of a foreign word is influenced by his native language knowledge, and such phenomenon is k…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionDesign of a novel Korean learning application for efficient pronunciation correction
The Korean wave, which denotes the global popularity of South Korea's cultural economy, contributes to the increasing demand for the Korean language. However, as there does not exist any application for foreigners to lea…
Sentencespeech-recognitionSpeech RecognitionSpeech-to-Text