Conformer-based Target-Speaker Automatic Speech Recognition for Single-Channel Audio
We propose CONF-TSASR, a non-autoregressive end-to-end time-frequency domain architecture for single-channel target-speaker automatic speech recognition (TS-ASR). The model consists of a TitaNet based speaker embedding module, a Conformer based masking as well as ASR modules. These modules are jointly optimized to transcribe a target-speaker, while ignoring speech from other speakers. For training we use Connectionist Temporal Classification (CTC) loss and introduce a scale-invariant spectrogram reconstruction loss to encourage the model better separate the target-speaker's spectrogram from mixture. We obtain state-of-the-art target-speaker word error rate (TS-WER) on WSJ0-2mix-extr (4.2%). Further, we report for the first time TS-WER on WSJ0-3mix-extr (12.4%), LibriSpeech2Mix (4.2%) and LibriSpeech3Mix (7.6%) datasets, establishing new benchmarks for TS-ASR. The proposed model will be open-sourced through NVIDIA NeMo toolkit.
Code (2)
Tasks
Automatic Speech Recognitionspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
A Conformer Based Acoustic Model for Robust Automatic Speech Recognition
This study addresses robust automatic speech recognition (ASR) by introducing a Conformer-based acoustic model. The proposed model builds on the wide residual bi-directional long short-term memory network (WRBN) with utt…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionContinual Learning for On-Device Speech Recognition using Disentangled Conformers
Automatic speech recognition research focuses on training and evaluating on static datasets. Yet, as speech models are increasingly deployed on personal devices, such models encounter user-specific distributional shifts.…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Continual LearningDomain Adaptation+2Confidence Score Based Conformer Speaker Adaptation for Speech Recognition
A key challenge for automatic speech recognition (ASR) systems is to model the speaker level variability. In this paper, compact speaker dependent learning hidden unit contributions (LHUC) are used to facilitate both spe…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Decoderspeech-recognition+1Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detection
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care to delay further progression. This paper presents the development of a state-of-the-art Conformer based speech recognition system bui…
Alzheimer's Disease DetectionData AugmentationNeural Architecture Searchspeech-recognition+1The RoyalFlush System of Speech Recognition for M2MeT Challenge
This paper describes our RoyalFlush system for the track of multi-speaker automatic speech recognition (ASR) in the M2MeT challenge. We adopted the serialized output training (SOT) based multi-speakers ASR system with la…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Data AugmentationDecoder+4