StressTransfer: Stress-Aware Speech-to-Speech Translation with Emphasis Preservation
We propose a stress-aware speech-to-speech translation (S2ST) system that preserves word-level emphasis by leveraging LLMs for cross-lingual emphasis conversion. Our method translates source-language stress into target-language tags that guide a controllable TTS model. To overcome data scarcity, we developed a pipeline to automatically generate aligned training data and introduce the "LLM-as-Judge" for evaluation. Experiments show our approach substantially outperforms baselines in preserving emphasis while maintaining comparable translation quality, speaker intent, and naturalness. Our work highlights the importance of prosody in translation and provides an effective, data-efficient solution for preserving paralinguistic cues in S2ST.
Code (0)
등록된 구현이 없습니다.
Tasks
Speech-to-Speech TranslationSimilar Papers 제목 키워드 기반
Evaluating and Preserving Lexical Stress in English-to-Chinese Speech-to-Speech Translation
Speech-to-speech translation (S2ST) systems have achieved impressive progress in semantic accuracy and speech naturalness. However, the cross-lingual transfer of lexical stress, a vital cue for emphasis and speaker inten…
Speech-to-Speech TranslationCross-Lingual TransferAttempt Towards Stress Transfer in Speech-to-Speech Machine Translation
The language diversity in India's education sector poses a significant challenge, hindering inclusivity. Despite the democratization of knowledge through online educational content, the dominance of English, as the inter…
DiversityMachine Translationtext-to-speechText to Speech+1Speech is More Than Words: Do Speech-to-Text Translation Systems Leverage Prosody?
The prosody of a spoken utterance, including features like stress, intonation and rhythm, can significantly affect the underlying semantics, and as a consequence can also affect its textual translation. Nevertheless, pro…
Rhythmspeech-recognitionSpeech RecognitionSpeech-to-Text+4Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark
Spoken meaning often depends not only on what is said, but also on which word is emphasized. The same sentence can convey correction, contrast, or clarification depending on where emphasis falls. Although modern text-to-…
Speech SynthesisRepresenting 'how you say' with 'what you say': English corpus of focused speech and text reflecting corresponding implications
In speech communication, how something is said (paralinguistic information) is as crucial as what is said (linguistic information). As a type of paralinguistic information, English speech uses sentence stress, the heavie…
SentenceTranslation