paper-with-me

홈 › Papers

How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?

2024-12-24 · Sara Papi, Peter Polak, Ondřej Bojar, Dominik Macháček

Simultaneous speech-to-text translation (SimulST) translates source-language speech into target-language text concurrently with the speaker's speech, ensuring low latency for better user comprehension. Despite its intended application to unbounded speech, most research has focused on human pre-segmented speech, simplifying the task and overlooking significant challenges. This narrow focus, coupled with widespread terminological inconsistencies, is limiting the applicability of research outcomes to real-world applications, ultimately hindering progress in the field. Our extensive literature review of 110 papers not only reveals these critical issues in current research but also serves as the foundation for our key contributions. We 1) define the steps and core components of a SimulST system, proposing a standardized terminology and taxonomy; 2) conduct a thorough analysis of community trends, and 3) offer concrete recommendations and future directions to bridge the gaps in existing literature, from evaluation frameworks to system architectures, for advancing the field towards more realistic and effective SimulST solutions.

📄 PDF Abstract BibTeX arXiv:2412.18495

Code (0)

등록된 구현이 없습니다.

Tasks

Simultaneous Speech-to-Text TranslationSpeech-to-TextSpeech-to-Text Translation

Similar Papers 제목 키워드 기반

Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voice

2025-07-23 · Shanbo Cheng, Yu Bao, Zhichao Huang, Yu Lu 외 arxiv

Simultaneous Interpretation (SI) represents one of the most daunting frontiers in the translation industry, with product-level automatic systems long plagued by intractable challenges: subpar transcription and translatio…

Speech-to-Speech TranslationReinforcement Learning

Adversarial speech for voice privacy protection from Personalized Speech generation

2024-01-22 · Shihao Chen, Liping Chen, Jie Zhang, KongAik Lee 외

The rapid progress in personalized speech generation technology, including personalized text-to-speech (TTS) and voice conversion (VC), poses a challenge in distinguishing between generated and real speech for human list…

Speaker Verificationtext-to-speechText to SpeechVoice Conversion

Vulnerability of Automatic Identity Recognition to Audio-Visual Deepfakes

2023-11-29 · Pavel Korshunov, Haolin Chen, Philip N. Garner, Sebastien Marcel

The task of deepfakes detection is far from being solved by speech or vision researchers. Several publicly available databases of fake synthetic video and speech were built to aid the development of detection methods. Ho…

Face RecognitionFace SwappingSpeaker Recognitiontext-to-speech+2

StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning

2024-06-05 · Shaolei Zhang, Qingkai Fang, Shoutao Guo, Zhengrui Ma 외

Simultaneous speech-to-speech translation (Simul-S2ST, a.k.a streaming speech translation) outputs target speech while receiving streaming speech inputs, which is critical for real-time communication. Beyond accomplishin…

Automatic Speech Recognition (ASR)de-enes-enfr-en+11

RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformer

2021-06-09 · Findings (ACL) 2021 8 · Xingshan Zeng, Liangyou Li, Qun Liu

End-to-end simultaneous speech translation (SST), which directly translates speech in one language into text in another language in real-time, is useful in many scenarios but has not been fully investigated. In this work…

RerankingTranslation