Enhancing Speech Intelligibility in Text-To-Speech Synthesis using Speaking Style Conversion
The increased adoption of digital assistants makes text-to-speech (TTS) synthesis systems an indispensable feature of modern mobile devices. It is hence desirable to build a system capable of generating highly intelligible speech in the presence of noise. Past studies have investigated style conversion in TTS synthesis, yet degraded synthesized quality often leads to worse intelligibility. To overcome such limitations, we proposed a novel transfer learning approach using Tacotron and WaveRNN based TTS synthesis. The proposed speech system exploits two modification strategies: (a) Lombard speaking style data and (b) Spectral Shaping and Dynamic Range Compression (SSDRC) which has been shown to provide high intelligibility gains by redistributing the signal energy on the time-frequency domain. We refer to this extension as Lombard-SSDRC TTS system. Intelligibility enhancement as quantified by the Intelligibility in Bits (SIIB-Gauss) measure shows that the proposed Lombard-SSDRC TTS system shows significant relative improvement between 110% and 130% in speech-shaped noise (SSN), and 47% to 140% in competing-speaker noise (CSN) against the state-of-the-art TTS approach. Additional subjective evaluation shows that Lombard-SSDRC TTS successfully increases the speech intelligibility with relative improvement of 455% for SSN and 104% for CSN in median keyword correction rate compared to the baseline TTS method.
Code (1)
Tasks
Speech Synthesistext-to-speechText to SpeechText-To-Speech SynthesisTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Vocal effort modeling in neural TTS for improving the intelligibility of synthetic speech in noise
We present a neural text-to-speech (TTS) method that models natural vocal effort variation to improve the intelligibility of synthetic speech in the presence of noise. The method consists of first measuring the spectral …
text-to-speechText to SpeechEvaluating Speech Synthesis by Training Recognizers on Synthetic Speech
Modern speech synthesis systems have improved significantly, with synthetic speech being indistinguishable from real speech. However, efficient and holistic evaluation of synthetic speech still remains a significant chal…
speech-recognitionSpeech RecognitionSpeech Synthesistext-to-speech+1Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model
Recent advances in generative language modeling applied to discrete speech tokens presented a new avenue for text-to-speech (TTS) synthesis. These speech language models (SLMs), similarly to their textual counterparts, a…
HallucinationLanguage ModelingLanguage ModellingSpeech Synthesis+3Vocoder-Based Speech Synthesis from Silent Videos
Both acoustic and visual information influence human perception of speech. For this reason, the lack of audio in a video sequence determines an extremely low speech intelligibility for untrained lip readers. In this pape…
Multi-Task LearningSpeech SynthesisSimple and Effective Unsupervised Speech Synthesis
We introduce the first unsupervised speech synthesis system based on a simple, yet effective recipe. The framework leverages recent work in unsupervised speech recognition as well as existing neural-based speech synthesi…
speech-recognitionSpeech RecognitionSpeech SynthesisUnsupervised Speech Recognition