Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation
A conventional approach to improving the performance of end-to-end speech translation (E2E-ST) models is to leverage the source transcription via pre-training and joint training with automatic speech recognition (ASR) and neural machine translation (NMT) tasks. However, since the input modalities are different, it is difficult to leverage source language text successfully. In this work, we focus on sequence-level knowledge distillation (SeqKD) from external text-based NMT models. To leverage the full potential of the source language information, we propose backward SeqKD, SeqKD from a target-to-source backward NMT model. To this end, we train a bilingual E2E-ST model to predict paraphrased transcriptions as an auxiliary task with a single decoder. The paraphrases are generated from the translations in bitext via back-translation. We further propose bidirectional SeqKD in which SeqKD from both forward and backward NMT models is combined. Experimental evaluations on both autoregressive and non-autoregressive models show that SeqKD in each direction consistently improves the translation performance, and the effectiveness is complementary regardless of the model capacity.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)DecoderKnowledge DistillationMachine TranslationNMTspeech-recognitionSpeech RecognitionTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification
End-to-end intent classification using speech has numerous advantages compared to the conventional pipeline approach using automatic speech recognition (ASR), followed by natural language processing modules. It attempts …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Classificationintent-classification+7End-to-End Speech Translation with Knowledge Distillation
End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years. Compared to conventional pipeline systems, end-to-e…
Knowledge Distillationspeech-recognitionSpeech RecognitionTranslationComparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR
Knowledge distillation is an effective machine learning technique to transfer knowledge from a teacher model to a smaller student model, especially with unlabeled data. In this paper, we focus on knowledge distillation f…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Knowledge Distillationspeech-recognition+2A Study of Non-autoregressive Model for Sequence Generation
Non-autoregressive (NAR) models generate all the tokens of a sequence in parallel, resulting in faster generation speed compared to their autoregressive (AR) counterparts but at the cost of lower accuracy. Different tech…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Knowledge DistillationMachine Translation+5Low-resource Low-footprint Wake-word Detection using Knowledge Distillation
As virtual assistants have become more diverse and specialized, so has the demand for application or brand-specific wake words. However, the wake-word-specific datasets typically used to train wake-word detectors are cos…
Knowledge Distillationspeech-recognitionSpeech RecognitionTransfer Learning