A GPU-based WFST Decoder with Exact Lattice Generation
We describe initial work on an extension of the Kaldi toolkit that supports weighted finite-state transducer (WFST) decoding on Graphics Processing Units (GPUs). We implement token recombination as an atomic GPU operation in order to fully parallelize the Viterbi beam search, and propose a dynamic load balancing strategy for more efficient token passing scheduling among GPU threads. We also redesign the exact lattice generation and lattice pruning algorithms for better utilization of the GPUs. Experiments on the Switchboard corpus show that the proposed method achieves identical 1-best results and lattice quality in recognition and confidence measure tasks, while running 3 to 15 times faster than the single process Kaldi decoder. The above results are reported on different GPU architectures. Additionally we obtain a 46-fold speedup with sequence parallelism and multi-process service (MPS) in GPU.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderGPUSchedulingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GPU-Accelerated Viterbi Exact Lattice Decoder for Batched Online and Offline Speech Recognition
We present an optimized weighted finite-state transducer (WFST) decoder capable of online streaming and offline batch processing of audio using Graphics Processing Units (GPUs). The decoder is efficient in memory utiliza…
CPUDecoderGPUspeech-recognition+1Integration of TensorFlow based Acoustic Model with Kaldi WFST Decoder
While the Kaldi framework provides state-of-the-art components for speech recognition like feature extraction, deep neural network (DNN)-based acoustic models, and a weighted finite state transducer (WFST)-based decoder,…
Decoderspeech-recognitionSpeech RecognitionWNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognition
Recently, attention-based encoder-decoder (AED) end-to-end (E2E) models have drawn more and more attention in the field of automatic speech recognition (ASR). AED models, however, still have drawbacks when deploying in c…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Decoderspeech-recognition+1Powerful and Extensible WFST Framework for RNN-Transducer Losses
This paper presents a framework based on Weighted Finite-State Transducers (WFST) to simplify the development of modifications for RNN-Transducer (RNN-T) loss. Existing implementations of RNN-T use CUDA-related code, whi…
Sequence-to-Lattice Models for Fast Translation
Non-autoregressive machine translation (NAT) approaches enable fast generation by utilizing parallelizable generative processes. The remaining bottleneck in these models is their decoder layers; unfortunately unlike in a…
DecoderMachine TranslationTranslation