paper-with-me

홈 › Papers

Does Simultaneous Speech Translation need Simultaneous Models?

2022-04-08 · Sara Papi, Marco Gaido, Matteo Negri, Marco Turchi

In simultaneous speech translation (SimulST), finding the best trade-off between high translation quality and low latency is a challenging task. To meet the latency constraints posed by the different application scenarios, multiple dedicated SimulST models are usually trained and maintained, generating high computational costs. In this paper, motivated by the increased social and environmental impact caused by these costs, we investigate whether a single model trained offline can serve not only the offline but also the simultaneous task without the need for any additional training or adaptation. Experiments on en->{de, es} indicate that, aside from facilitating the adoption of well-established offline techniques and architectures without affecting latency, the offline solution achieves similar or better translation quality compared to the same model trained in simultaneous settings, as well as being competitive with the SimulST state of the art.

📄 PDF Abstract BibTeX arXiv:2204.03783

Code (1)

hlt-mt/fbk-fairseq 공식 구현 pytorch

Tasks

Translation

Similar Papers 제목 키워드 기반

Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems

2021-10-13 · Mohd Abbas Zaidi, Beomseok Lee, Sangha Kim, Chanwoo Kim

Simultaneous translation systems start producing the output while processing the partial source sentence in the incoming input stream. These systems need to decide when to read more input and when to write the output. Th…

SentenceSimultaneous Speech-to-Text TranslationSpeech-to-TextSpeech-to-Text Translation+1

Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training

2020-10-20 · Findings of the Association for Computational Linguistics 2020 · Renjie Zheng, Mingbo Ma, Baigong Zheng, Kaibo Liu 외

Simultaneous speech-to-speech translation is widely useful but extremely challenging, since it needs to generate target-language speech concurrently with the source-language speech, with only a few seconds delay. In addi…

SentenceSimultaneous Speech-to-Speech TranslationSpeech-to-Speech TranslationTranslation

CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022

2022-04-12 · IWSLT (ACL) 2022 5 · Peter Polák, Ngoc-Quan Ngoc, Tuan-Nam Nguyen, Danni Liu 외

In this paper, we describe our submission to the Simultaneous Speech Translation at IWSLT 2022. We explore strategies to utilize an offline model in a simultaneous setting without the need to modify the original model. I…

Translation

SimulSpeech: End-to-End Simultaneous Speech to Text Translation

2020-07-01 · ACL 2020 6 · Yi Ren, Jinglin Liu, Xu Tan, Chen Zhang 외

In this work, we develop SimulSpeech, an end-to-end simultaneous speech to text translation system which translates speech in source language to text in target language concurrently. SimulSpeech consists of a speech enco…

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)DecoderKnowledge Distillation+9

What does it take to get state of the art in simultaneous speech-to-speech translation?

2024-09-02 · Vincent Wilmet, Johnson Du

This paper presents an in-depth analysis of the latency characteristics observed in simultaneous speech-to-speech model's performance, particularly focusing on hallucination-induced latency spikes. By systematically expe…

HallucinationManagementSimultaneous Speech-to-Speech TranslationSpeech-to-Speech Translation