Adversarial Feature Learning and Unsupervised Clustering based Speech Synthesis for Found Data with Acoustic and Textual Noise

2020-04-28 · Shan Yang, Yuxuan Wang, Lei Xie

Attention-based sequence-to-sequence (seq2seq) speech synthesis has achieved extraordinary performance. But a studio-quality corpus with manual transcription is necessary to train such seq2seq systems. In this paper, we propose an approach to build high-quality and stable seq2seq based speech synthesis system using challenging found data, where training speech contains noisy interferences (acoustic noise) and texts are imperfect speech recognition transcripts (textual noise). To deal with text-side noise, we propose a VQVAE based heuristic method to compensate erroneous linguistic feature with phonetic information learned directly from speech. As for the speech-side noise, we propose to learn a noise-independent feature in the auto-regressive decoder through adversarial training and data augmentation, which does not need an extra speech enhancement model. Experiments show the effectiveness of the proposed approach in dealing with text-side and speech-side noise. Surpassing the denoising approach based on a state-of-the-art speech enhancement model, our system built on noisy found data can synthesize clean and high-quality speech with MOS close to the system built on the clean counterpart.

📄 PDF Abstract BibTeX arXiv:2004.13595

Code (0)

등록된 구현이 없습니다.

Tasks

ClusteringData Augmentation Decoder Denoising Speech Enhancement speech-recognition Speech Recognition Speech Synthesis

Methods 이 논문이 사용한 방법론

VQ-VAE VQ-VAE is a type of variational autoencoder that uses vector quantisation to obtain a discrete latent representation. It differs from…

Controllable speech synthesis by learning discrete phoneme-level prosodic representations

2022-11-29 · Nikolaos Ellinas, Myrsini Christidou, Alexandra Vioni, June Sig Sung 외

In this paper, we present a novel method for phoneme-level prosody control of F0 and duration using intuitive discrete labels. We propose an unsupervised prosodic clustering process which is used to discretize phoneme-le…

Clustering Speech Synthesis text-to-speech Text to Speech

Prosodic Clustering for Phoneme-level Prosody Control in End-to-End Speech Synthesis

2021-11-19 · Alexandra Vioni, Myrsini Christidou, Nikolaos Ellinas, Georgios Vamvoukakis 외

This paper presents a method for controlling the prosody at the phoneme level in an autoregressive attention-based text-to-speech system. Instead of learning latent prosodic features with a variational framework as is co…

Clustering Decoder Speech Synthesis text-to-speech+1

Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling

2019-06-17 · Siyuan Feng, Tan Lee, Zhiyuan Peng

This study addresses the problem of unsupervised subword unit discovery from untranscribed speech. It forms the basis of the ultimate goal of ZeroSpeech 2019, building text-to-speech systems without text labels. In this …

Representation Learning Speech Representation Learning text-to-speech Text to Speech

Unsupervised Style and Content Separation by Minimizing Mutual Information for Speech Synthesis

2020-03-09 · Ting-yao Hu, Ashish Shrivastava, Oncel Tuzel, Chandra Dhir

We present a method to generate speech from input text and a style vector that is extracted from a reference speech signal in an unsupervised manner, i.e., no style annotation, such as speaker information, is required. E…

Decoder Speech Synthesis

Deep Denoising Auto-encoder for Statistical Speech Synthesis

2015-06-17 · Zhenzhou Wu, Shinji Takaki, Junichi Yamagishi

This paper proposes a deep denoising auto-encoder technique to extract better acoustic features for speech synthesis. The technique allows us to automatically extract low-dimensional features from high dimensional spectr…

Denoising Speech Synthesis text-to-speech Text to Speech