paper-with-me

홈 › Papers

Contextual Expressive Text-to-Speech

2022-11-26 · Jianhong Tu, Zeyu Cui, Xiaohuan Zhou, Siqi Zheng, Kai Hu, Ju Fan, Chang Zhou

The goal of expressive Text-to-speech (TTS) is to synthesize natural speech with desired content, prosody, emotion, or timbre, in high expressiveness. Most of previous studies attempt to generate speech from given labels of styles and emotions, which over-simplifies the problem by classifying styles and emotions into a fixed number of pre-defined categories. In this paper, we introduce a new task setting, Contextual TTS (CTTS). The main idea of CTTS is that how a person speaks depends on the particular context she is in, where the context can typically be represented as text. Thus, in the CTTS task, we propose to utilize such context to guide the speech synthesis process instead of relying on explicit labels of styles and emotions. To achieve this task, we construct a synthetic dataset and develop an effective framework. Experiments show that our framework can generate high-quality expressive speech based on the given context both in synthetic datasets and real-world scenarios.

📄 PDF Abstract BibTeX arXiv:2211.14548

Code (0)

등록된 구현이 없습니다.

Tasks

Speech Synthesistext-to-speechText to Speech

Similar Papers 제목 키워드 기반

DiffCSS: Diverse and Expressive Conversational Speech Synthesis with Diffusion Models

2025-02-27 · Weihao wu, Zhiwei Lin, Yixuan Zhou, Jingbei Li 외

Conversational speech synthesis (CSS) aims to synthesize both contextually appropriate and expressive speech, and considerable efforts have been made to enhance the understanding of conversational context. However, exist…

DiversityLanguage ModelingLanguage ModellingSpeech Synthesis

Prompt-Unseen-Emotion: Zero-shot Expressive Speech Synthesis with Prompt-LLM Contextual Knowledge for Mixed Emotions

2025-06-03 · Xiaoxue Gao, Huayun Zhang, Nancy F. Chen

Existing expressive text-to-speech (TTS) systems primarily model a limited set of categorical emotions, whereas human conversations extend far beyond these predefined emotions, making it essential to explore more diverse…

Expressive Speech SynthesisPrompt LearningSpeech Synthesistext-to-speech+1

Computational Narrative Understanding for Expressive Text-to-Speech

2025-09-04 · Gaspard Michel, Elena V. Epure, Christophe Cerisara arxiv

Recent advances in text-to-speech (TTS) have been driven by large, multi-domain speech corpora, yet the expressive potential of audiobook data remains underexamined. We argue that human-narrated audiobooks, particularly …

Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation

2021-12-04 · Yingruo Fan, Zhaojiang Lin, Jun Saito, Wenping Wang 외

Speech-driven 3D facial animation with accurate lip synchronization has been widely studied. However, synthesizing realistic motions for the entire face during speech has rarely been explored. In this work, we present a …

Language Modelling

Towards Expressive Speaking Style Modelling with Hierarchical Context Information for Mandarin Speech Synthesis

2022-03-23 · Shun Lei, Yixuan Zhou, Liyang Chen, Zhiyong Wu 외

Previous works on expressive speech synthesis mainly focus on current sentence. The context in adjacent sentences is neglected, resulting in inflexible speaking style for the same text, which lacks speech variations. In …

Expressive Speech SynthesisKnowledge DistillationSentenceSpeech Synthesis