On Knowledge Distillation for Translating Erroneous Speech Transcriptions
Recent studies argue that knowledge distillation is promising for speech translation (ST) using end-to-end models. In this work, we investigate the effect of knowledge distillation with a cascade ST using automatic speech recognition (ASR) and machine translation (MT) models. We distill knowledge from a teacher model based on human transcripts to a student model based on erroneous transcriptions. Our experimental results demonstrated that knowledge distillation is beneficial for a cascade ST. Further investigation that combined knowledge distillation and fine-tuning revealed that the combination consistently improved two language pairs: English-Italian and Spanish-English.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Knowledge DistillationMachine Translationspeech-recognitionSpeech RecognitionTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving End-to-End Speech Translation by Imitation-Based Knowledge Distillation with Synthetic Transcripts
End-to-end automatic speech translation (AST) relies on data that combines audio inputs with text translation outputs. Previous work used existing large parallel corpora of transcriptions and translations in a knowledge …
automatic-speech-translationImitation LearningKnowledge DistillationMachine Translation+3Listen Like a Teacher: Mitigating Whisper Hallucinations using Adaptive Layer Attention and Knowledge Distillation
The Whisper model, an open-source automatic speech recognition system, is widely adopted for its strong performance across multilingual and zero-shot settings. However, it frequently suffers from hallucination errors, es…
Knowledge DistillationSpeech RecognitionMulti-Task Modeling of Phonographic Languages: Translating Middle Egyptian Hieroglyphs
Machine translation of ancient languages faces a low-resource problem, caused by the limited amount of available textual source data and their translations. We present a multi-task modeling approach to translating Middle…
Machine TranslationMulti-Task LearningPOSPOS Tagging+3Topic Model Robustness to Automatic Speech Recognition Errors in Podcast Transcripts
For a multilingual podcast streaming service, it is critical to be able to deliver relevant content to all users independent of language. Podcast content relevance is conventionally determined using various metadata sour…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionBridging Modalities: Knowledge Distillation and Masked Training for Translating Multi-Modal Emotion Recognition to Uni-Modal, Speech-Only Emotion Recognition
This paper presents an innovative approach to address the challenges of translating multi-modal emotion recognition models to a more practical and resource-efficient uni-modal counterpart, specifically focusing on speech…
Emotion RecognitionKnowledge Distillation