Phonological Features for 0-shot Multilingual Speech Synthesis
Code-switching---the intra-utterance use of multiple languages---is prevalent across the world. Within text-to-speech (TTS), multilingual models have been found to enable code-switching. By modifying the linguistic input to sequence-to-sequence TTS, we show that code-switching is possible for languages unseen during training, even within monolingual models. We use a small set of phonological features derived from the International Phonetic Alphabet (IPA), such as vowel height and frontness, consonant place and manner. This allows the model topology to stay unchanged for different languages, and enables new, previously unseen feature combinations to be interpreted by the model. We show that this allows us to generate intelligible, code-switched speech in a new language at test time, including the approximation of sounds never seen in training.
Code (1)
Tasks
Speech Synthesistext-to-speechText to SpeechSimilar Papers 제목 키워드 기반
Multilingual and crosslingual speech recognition using phonological-vector based phone embeddings
The use of phonological features (PFs) potentially allows language-specific phones to remain linked in training, which is highly desirable for information sharing for multilingual and crosslingual speech recognition meth…
speech-recognitionSpeech RecognitionCross-lingual Low Resource Speaker Adaptation Using Phonological Features
The idea of using phonological features instead of phonemes as input to sequence-to-sequence TTS has been recently proposed for zero-shot multilingual speech synthesis. This approach is useful for code-switching, as it f…
Speech SynthesisApplying Feature Underspecified Lexicon Phonological Features in Multilingual Text-to-Speech
This study investigates whether the phonological features derived from the Featurally Underspecified Lexicon model can be applied in text-to-speech systems to generate native and non-native speech in English and Mandarin…
Language Acquisitiontext-to-speechText to SpeechApplying Phonological Features in Multilingual Text-To-Speech
This study investigates whether phonological features can be applied in text-to-speech systems to generate native and non-native speech in English and Mandarin. We present a mapping of ARPABET/pinyin to SAMPA/SAMPA-SC an…
Language Acquisitiontext-to-speechText to SpeechMultilingual Phonological Feature Recognition with Self-Supervised Speech Models
Phonological features provide a language-general and linguistically grounded representation of speech. We present PhonoQ-2.0, a multilingual frame-level phonological feature recognizer built on self-supervised speech mod…