Accelerating NMT Batched Beam Decoding with LMBR Posteriors for Deployment
We describe a batched beam decoding algorithm for NMT with LMBR n-gram posteriors, showing that LMBR techniques still yield gains on top of the best recently reported results with Transformers. We also discuss acceleration strategies for deployment, and the effect of the beam size and batching on memory and speed.
Code (0)
등록된 구현이 없습니다.
Tasks
NMTSimilar Papers 제목 키워드 기반
A Streaming Approach For Efficient Batched Beam Search
We propose an efficient batching strategy for variable-length decoding on GPU architectures. During decoding, when candidates terminate or are pruned according to heuristics, our streaming approach periodically "refills"…
GPUMachine TranslationTranslationBeam Search Decoding using Manner of Articulation Detection Knowledge Derived from Connectionist Temporal Classification
Manner of articulation detection using deep neural networks require a priori knowledge of the attribute discriminative features or the decent phoneme alignments. However generating an appropriate phoneme alignment is com…
AttributeGeneral ClassificationManner Of Articulation DetectionFlexCTC: GPU-powered CTC Beam Decoding With Advanced Contextual Abilities
While beam search improves speech recognition quality over greedy decoding, standard implementations are slow, often sequential, and CPU-bound. To fully leverage modern hardware capabilities, we present a novel open-sour…
Speech RecognitionWIND: Accelerated RNN-T Decoding with Windowed Inference for Non-blank Detection
We propose Windowed Inference for Non-blank Detection (WIND), a novel strategy that significantly accelerates RNN-T inference without compromising model accuracy. During model inference, instead of processing frames sequ…
TurboBias 2.0: Streaming Context-Biasing for Production-Efficient ASR Systems
Contextualization is essential for production automatic speech recognition (ASR) systems, where user-provided phrases must be recognized accurately under strict latency constraints. Although many context-biasing methods …
Speech Recognition