Does Simultaneous Speech Translation need Simultaneous Models?
In simultaneous speech translation (SimulST), finding the best trade-off between high translation quality and low latency is a challenging task. To meet the latency constraints posed by the different application scenarios, multiple dedicated SimulST models are usually trained and maintained, generating high computational costs. In this paper, motivated by the increased social and environmental impact caused by these costs, we investigate whether a single model trained offline can serve not only the offline but also the simultaneous task without the need for any additional training or adaptation. Experiments on en->{de, es} indicate that, aside from facilitating the adoption of well-established offline techniques and architectures without affecting latency, the offline solution achieves similar or better translation quality compared to the same model trained in simultaneous settings, as well as being competitive with the SimulST state of the art.
Code (1)
Tasks
TranslationSimilar Papers 제목 키워드 기반
Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems
Simultaneous translation systems start producing the output while processing the partial source sentence in the incoming input stream. These systems need to decide when to read more input and when to write the output. Th…
SentenceSimultaneous Speech-to-Text TranslationSpeech-to-TextSpeech-to-Text Translation+1Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training
Simultaneous speech-to-speech translation is widely useful but extremely challenging, since it needs to generate target-language speech concurrently with the source-language speech, with only a few seconds delay. In addi…
SentenceSimultaneous Speech-to-Speech TranslationSpeech-to-Speech TranslationTranslationCUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022
In this paper, we describe our submission to the Simultaneous Speech Translation at IWSLT 2022. We explore strategies to utilize an offline model in a simultaneous setting without the need to modify the original model. I…
TranslationSimulSpeech: End-to-End Simultaneous Speech to Text Translation
In this work, we develop SimulSpeech, an end-to-end simultaneous speech to text translation system which translates speech in source language to text in target language concurrently. SimulSpeech consists of a speech enco…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)DecoderKnowledge Distillation+9What does it take to get state of the art in simultaneous speech-to-speech translation?
This paper presents an in-depth analysis of the latency characteristics observed in simultaneous speech-to-speech model's performance, particularly focusing on hallucination-induced latency spikes. By systematically expe…
HallucinationManagementSimultaneous Speech-to-Speech TranslationSpeech-to-Speech Translation