CrossSpeech: Speaker-independent Acoustic Representation for Cross-lingual Speech Synthesis
While recent text-to-speech (TTS) systems have made remarkable strides toward human-level quality, the performance of cross-lingual TTS lags behind that of intra-lingual TTS. This gap is mainly rooted from the speaker-language entanglement problem in cross-lingual TTS. In this paper, we propose CrossSpeech which improves the quality of cross-lingual speech by effectively disentangling speaker and language information in the level of acoustic feature space. Specifically, CrossSpeech decomposes the speech generation pipeline into the speaker-independent generator (SIG) and speaker-dependent generator (SDG). The SIG produces the speaker-independent acoustic representation which is not biased to specific speaker distributions. On the other hand, the SDG models speaker-dependent speech variation that characterizes speaker attributes. By handling each information separately, CrossSpeech can obtain disentangled speaker and language representations. From the experiments, we verify that CrossSpeech achieves significant improvements in cross-lingual TTS, especially in terms of speaker similarity to the target speaker.
Code (0)
등록된 구현이 없습니다.
Tasks
Speech Synthesistext-to-speechText to SpeechMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CrossSpeech++: Cross-lingual Speech Synthesis with Decoupled Language and Speaker Generation
The goal of this work is to generate natural speech in multiple languages while maintaining the same speaker identity, a task known as cross-lingual speech synthesis. A key challenge of cross-lingual speech synthesis is …
Speech SynthesisSpeaker-Independent Acoustic-to-Articulatory Inversion through Multi-Channel Attention Discriminator
We present a novel speaker-independent acoustic-to-articulatory inversion (AAI) model, overcoming the limitations observed in conventional AAI models that rely on acoustic features derived from restricted datasets. To ad…
Self-Supervised LearningSpeaker-Invariant Training via Adversarial Learning
We propose a novel adversarial multi-task learning scheme, aiming at actively curtailing the inter-talker feature variability while maximizing its senone discriminability so as to enhance the performance of a deep neural…
General ClassificationMulti-Task LearningSEAL: Speaker Error Correction using Acoustic-conditioned Large Language Models
Speaker Diarization (SD) is a crucial component of modern end-to-end ASR pipelines. Traditional SD systems, which are typically audio-based and operate independently of ASR, often introduce speaker errors, particularly d…
speaker-diarizationSpeaker DiarizationSpeaker-independent raw waveform model for glottal excitation
Recent speech technology research has seen a growing interest in using WaveNets as statistical vocoders, i.e., generating speech waveforms from acoustic features. These models have been shown to improve the generated spe…
modelSpeech Synthesistext-to-speechText to Speech+2