Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation
Recent works in spoken language translation (SLT) have attempted to build end-to-end speech-to-text translation without using source language transcription during learning or decoding. However, while large quantities of parallel texts (such as Europarl, OpenSubtitles) are available for training machine translation systems, there are no large (100h) and open source parallel corpora that include speech in a source language aligned to text in a target language. This paper tries to fill this gap by augmenting an existing (monolingual) corpus: LibriSpeech. This corpus, used for automatic speech recognition, is derived from read audiobooks from the LibriVox project, and has been carefully segmented and aligned. After gathering French e-books corresponding to the English audio-books from LibriSpeech, we align speech segments at the sentence level with their respective translations and obtain 236h of usable parallel data. This paper presents the details of the processing as well as a manual evaluation conducted on a small subset of the corpus. This evaluation shows that the automatic alignments scores are reasonably correlated with the human judgments of the bilingual alignment quality. We believe that this corpus (which is made available online) is useful for replicable experiments in direct speech translation or more general spoken language translation experiments.
Code (1)
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Machine TranslationSentencespeech-recognitionSpeech RecognitionSpeech-to-TextSpeech-to-Text TranslationTranslationSimilar Papers 제목 키워드 기반
A corpus of post-edited translations (Un corpus d'erreurs de traduction) [in French]
Microsoft Speech Language Translation (MSLT) Corpus: The IWSLT 2016 release for English, French and German
We describe the Microsoft Speech Language Translation (MSLT) corpus, which was created in order to evaluate end-to-end conversational speech translation quality. The corpus was created from actual conversations over Skyp…
Machine Translationspeech-recognitionSpeech RecognitionTranslationPantagruel: Unified Self-Supervised Encoders for French Text and Speech
We release Pantagruel models, a new family of self-supervised encoder models for French text and speech. Instead of predicting modality-tailored targets such as textual tokens or speech units, Pantagruel learns contextua…
Representation LearningComparing Formulaic Language in Human and Machine Translation: Insight from a Parliamentary Corpus
A recent study has shown that, compared to human translations, neural machine translations contain more strongly-associated formulaic sequences made of relatively high-frequency words, but far less strongly-associated fo…
ArticlesMachine TranslationTranslationRosetta-LSF: an Aligned Corpus of French Sign Language and French for Text-to-Sign Translation
This article presents a new French Sign Language (LSF) corpus called “Rosetta-LSF”. It was created to support future studies on the automatic translation of written French into LSF, rendered through the animation of a vi…