T3M: Text Guided 3D Human Motion Synthesis from Speech
Speech-driven 3D motion synthesis seeks to create lifelike animations based on human speech, with potential uses in virtual reality, gaming, and the film production. Existing approaches reply solely on speech audio for motion generation, leading to inaccurate and inflexible synthesis results. To mitigate this problem, we introduce a novel text-guided 3D human motion synthesis method, termed \textit{T3M}. Unlike traditional approaches, T3M allows precise control over motion synthesis via textual input, enhancing the degree of diversity and user customization. The experiment results demonstrate that T3M can greatly outperform the state-of-the-art methods in both quantitative metrics and qualitative evaluations. We have publicly released our code at \href{https://github.com/Gloria2tt/T3M.git}{https://github.com/Gloria2tt/T3M.git}
Code (1)
Tasks
DiversityMotion GenerationMotion SynthesisSimilar Papers 제목 키워드 기반
Prompt-Unseen-Emotion: Zero-shot Expressive Speech Synthesis with Prompt-LLM Contextual Knowledge for Mixed Emotions
Existing expressive text-to-speech (TTS) systems primarily model a limited set of categorical emotions, whereas human conversations extend far beyond these predefined emotions, making it essential to explore more diverse…
Expressive Speech SynthesisPrompt LearningSpeech Synthesistext-to-speech+1AuEmoChat: Authentic Emotion Understanding and Rendering for Conversational Speech Synthesis
Conversational Speech Synthesis (CSS) aims to synthesize speech with human-like emotional expression and contextual consistency in user-agent interactions. Existing CSS methods struggle to render authentic human emotions…
Speech SynthesisDeepGesture: A conversational gesture synthesis system based on emotions and semantics
Along with the explosion of large language models, improvements in speech synthesis, advancements in hardware, and the evolution of computer graphics, the current bottleneck in creating digital humans lies in generating …
Gesture GenerationMotion SynthesisSpeech SynthesisUnityEmoInstruct-TTS: Dual-Path Instruction-Guided Emotional Speech Synthesis
Instruction-based controllable speech synthesis enables users to specify emotions through natural language. However, existing approaches often rely on coarse emotion labels and lack explicit modeling of fine-grained inte…
Speech SynthesisPROEMO: Prompt-Driven Text-to-Speech Synthesis Based on Emotion and Intensity Control
Speech synthesis has significantly advanced from statistical methods to deep neural network architectures, leading to various text-to-speech (TTS) models that closely mimic human speech patterns. However, capturing nuanc…
Speech Synthesistext-to-speechText to SpeechText-To-Speech Synthesis