Blank-regularized CTC for Frame Skipping in Neural Transducer
Neural Transducer and connectionist temporal classification (CTC) are popular end-to-end automatic speech recognition systems. Due to their frame-synchronous design, blank symbols are introduced to address the length mismatch between acoustic frames and output tokens, which might bring redundant computation. Previous studies managed to accelerate the training and inference of neural Transducers by discarding frames based on the blank symbols predicted by a co-trained CTC. However, there is no guarantee that the co-trained CTC can maximize the ratio of blank symbols. This paper proposes two novel regularization methods to explicitly encourage more blanks by constraining the self-loop of non-blank symbols in the CTC. It is interesting to find that the frame reduction ratio of the neural Transducer can approach the theoretical boundary. Experiments on LibriSpeech corpus show that our proposed method accelerates the inference of neural Transducer by 4 times without sacrificing performance. Our work is open-sourced and publicly available https://github.com/k2-fsa/icefall.
Code (1)
Tasks
Automatic Speech Recognitionspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Factorized Blank Thresholding for Improved Runtime Efficiency of Neural Transducers
We show how factoring the RNN-T's output distribution can significantly reduce the computation cost and power consumption for on-device ASR inference with no loss in accuracy. With the rise in popularity of neural-transd…
Lightweight Transducer Based on Frame-Level Criterion
The transducer model trained based on sequence-level criterion requires a lot of memory due to the generation of the large probability matrix. We proposed a lightweight transducer model based on frame-level criterion, wh…
Decoderimbalanced classificationSpeech RecognitionFSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularization
Transducer-based models, such as RNN-Transducer and transformer-transducer, have achieved great success in speech recognition. A typical transducer model decodes the output sequence conditioned on the current acoustic st…
DecoderPositionspeech-recognitionSpeech RecognitionMulti-blank Transducers for Speech Recognition
This paper proposes a modification to RNN-Transducer (RNN-T) models for automatic speech recognition (ASR). In standard RNN-T, the emission of a blank symbol consumes exactly one input frame; in our proposed method, we i…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionCTC Blank Triggered Dynamic Layer-Skipping for Efficient CTC-based Speech Recognition
Deploying end-to-end speech recognition models with limited computing resources remains challenging, despite their impressive performance. Given the gradual increase in model size and the wide range of model applications…
Knowledge Distillationspeech-recognitionSpeech Recognition