Knowledge Distillation for Neural Transducer-based Target-Speaker ASR: Exploiting Parallel Mixture/Single-Talker Speech Data
Neural transducer (RNNT)-based target-speaker speech recognition (TS-RNNT) directly transcribes a target speaker's voice from a multi-talker mixture. It is a promising approach for streaming applications because it does not incur the extra computation costs of a target speech extraction frontend, which is a critical barrier to quick response. TS-RNNT is trained end-to-end given the input speech (i.e., mixtures and enrollment speech) and reference transcriptions. The training mixtures are generally simulated by mixing single-talker signals, but conventional TS-RNNT training does not utilize single-speaker signals. This paper proposes using knowledge distillation (KD) to exploit the parallel mixture/single-talker speech data. Our proposed KD scheme uses an RNNT system pretrained with the target single-talker speech input to generate pseudo labels for the TS-RNNT training. Experimental results show that TS-RNNT systems trained with the proposed KD scheme outperform a baseline TS-RNNT.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationSpeech Extractionspeech-recognitionSpeech RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
BECTRA: Transducer-based End-to-End ASR with BERT-Enhanced Encoder
We present BERT-CTC-Transducer (BECTRA), a novel end-to-end automatic speech recognition (E2E-ASR) model formulated by the transducer with a BERT-enhanced encoder. Integrating a large-scale pre-trained language model (LM…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)DecoderLanguage Modelling+2Knowledge Distillation for Neural Transducers from Large Self-Supervised Pre-trained Models
Self-supervised pre-training is an effective approach to leveraging a large amount of unlabelled data to reduce word error rates (WERs) of automatic speech recognition (ASR) systems. Since it is impractical to use large …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Knowledge DistillationLanguage Modelling+2CoDERT: Distilling Encoder Representations with Co-learning for Transducer-based Speech Recognition
We propose a simple yet effective method to compress an RNN-Transducer (RNN-T) through the well-known knowledge distillation paradigm. We show that the transducer's encoder outputs naturally have a high entropy and conta…
DecoderKnowledge Distillationspeech-recognitionSpeech RecognitionReducing the gap between streaming and non-streaming Transducer-based ASR by adaptive two-stage knowledge distillation
Transducer is one of the mainstream frameworks for streaming speech recognition. There is a performance gap between the streaming and non-streaming transducer models due to limited context. To reduce this gap, an effecti…
Knowledge Distillationspeech-recognitionSpeech RecognitionTurn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection
In this paper, we present a novel speaker diarization system for streaming on-device applications. In this system, we use a transformer transducer to detect the speaker turns, represent each speaker turn by a speaker emb…
Clusteringspeaker-diarizationSpeaker Diarization