Direct Segmentation Models for Streaming Speech Translation
The cascade approach to Speech Translation (ST) is based on a pipeline that concatenates an Automatic Speech Recognition (ASR) system followed by a Machine Translation (MT) system. These systems are usually connected by a segmenter that splits the ASR output into hopefully, semantically self-contained chunks to be fed into the MT system. This is specially challenging in the case of streaming ST, where latency requirements must also be taken into account. This work proposes novel segmentation models for streaming ST that incorporate not only textual, but also acoustic information to decide when the ASR output is split into a chunk. An extensive and throughly experimental setup is carried out on the Europarl-ST dataset to prove the contribution of acoustic information to the performance of the segmentation model in terms of BLEU score in a streaming ST scenario. Finally, comparative results with previous work also show the superiority of the segmentation models proposed in this work.
Code (1)
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Machine TranslationSegmentationspeech-recognitionSpeech RecognitionTranslationSimilar Papers 제목 키워드 기반
End-to-End Simultaneous Speech Translation with Differentiable Segmentation
End-to-end simultaneous speech translation (SimulST) outputs translation while receiving the streaming speech inputs (a.k.a. streaming speech translation), and hence needs to segment the speech inputs and then translate …
SegmentationTranslationLearning Adaptive Segmentation Policy for End-to-End Simultaneous Translation
End-to-end simultaneous speech-to-text translation aims to directly perform translation from streaming source speech to target text with high translation quality and low latency. A typical simultaneous translation (ST) s…
SegmentationSimultaneous Speech-to-Text TranslationSpeech-to-TextSpeech-to-Text Translation+1Learning When to Translate for Streaming Speech
How to find proper moments to generate partial sentence translation given a streaming speech input? Existing approaches waiting-and-translating for a fixed duration often break the acoustic units in speech, since the bou…
DecoderSentenceSpeech-to-Text TranslationTranslationStreaming Punctuation: A Novel Punctuation Technique Leveraging Bidirectional Context for Continuous Speech Recognition
While speech recognition Word Error Rate (WER) has reached human parity for English, continuous speech recognition scenarios such as voice typing and meeting transcriptions still suffer from segmentation and punctuation …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Machine TranslationSegmentation+2