Tied Multitask Learning for Neural Speech Translation
We explore multitask models for neural translation of speech, augmenting them in order to reflect two intuitive notions. First, we introduce a model where the second task decoder receives information from the decoder of the first task, since higher-level intermediate representations should provide useful information. Second, we apply regularization that encourages transitivity and invertibility. We show that the application of these notions on jointly trained models improves performance on the tasks of low-resource speech transcription and translation. It also leads to better performance when using attention information for word discovery over unsegmented input.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderTranslationSimilar Papers 제목 키워드 기반
MSLM-S2ST: A Multitask Speech Language Model for Textless Speech-to-Speech Translation with Speaker Style Preservation
There have been emerging research interest and advances in speech-to-speech translation (S2ST), translating utterances from one language to another. This work proposes Multitask Speech Language Model (MSLM), which is a d…
DecoderLanguage ModelingLanguage ModellingSpeech-to-Speech Translation+1Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation
Speech translation (ST) aims to learn transformations from speech in the source language to the text in the target language. Previous works show that multitask learning improves the ST performance, in which the recogniti…
DecoderTranslationImproving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task
Pretraining and multitask learning are widely used to improve the speech to text translation performance. In this study, we are interested in training a speech to text translation model along with an auxiliary text to te…
DecoderKnowledge DistillationSpeech-to-TextSpeech-to-Text Translation+2Exploring Universal Speech Attributes for Speaker Verification with an Improved Cross-stitch Network
The universal speech attributes for x-vector based speaker verification (SV) are addressed in this paper. The manner and place of articulation form the fundamental speech attribute unit (SAU), and then new speech attribu…
AttributeSpeaker VerificationThe University of Helsinki Submission to the IWSLT2020 Offline SpeechTranslation Task
This paper describes the University of Helsinki Language Technology group{'}s participation in the IWSLT 2020 offline speech translation task, addressing the translation of English audio into German text. In line with th…
Transfer LearningTranslation