Improving RNN-Transducers with Acoustic LookAhead
RNN-Transducers (RNN-Ts) have gained widespread acceptance as an end-to-end model for speech to text conversion because of their high accuracy and streaming capabilities. A typical RNN-T independently encodes the input audio and the text context, and combines the two encodings by a thin joint network. While this architecture provides SOTA streaming accuracy, it also makes the model vulnerable to strong LM biasing which manifests as multi-step hallucination of text without acoustic evidence. In this paper we propose LookAhead that makes text representations more acoustically grounded by looking ahead into the future within the audio input. This technique yields a significant 5%-20% relative reduction in word error rate on both in-domain and out-of-domain evaluation sets.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationSpeech-to-TextSimilar Papers 제목 키워드 기반
A Novel Acoustic Wearable for Assessment of Tendon Health and Loading Condition
The current methods of assessing tendon health such as clinical examination, imaging techniques, and implanted pressure sensors, are often based on a subjective assessment or are not accurate enough, are extremely expens…
Structure-Aware Path Inference for Neural Finite State Transducers
Neural finite-state transducers (NFSTs) form an expressive family of neurosymbolic sequence transduction models. An NFST models each string pair as having been generated by a latent path in a finite-state transducer. As …
Simulating room transfer functions between transducers mounted on audio devices using a modified image source method
The image source method (ISM) is often used to simulate room acoustics due to its ease of use and computational efficiency. The standard ISM is limited to simulations of room impulse responses between point sources and o…
Computational Efficiencyparameter estimationSpeech EnhancementModel-Based Photoacoustic Image Reconstruction using Compressed Sensing and Smoothed L0 Norm
Photoacoustic imaging (PAI) is a novel medical imaging modality that uses the advantages of the spatial resolution of ultrasound imaging and the high contrast of pure optical imaging. Analytical algorithms are usually em…
compressed sensingImage ReconstructionUtilizing Neural Transducers for Two-Stage Text-to-Speech via Semantic Token Prediction
We propose a novel text-to-speech (TTS) framework centered around a neural transducer. Our approach divides the whole TTS pipeline into semantic-level sequence-to-sequence (seq2seq) modeling and fine-grained acoustic mod…
text-to-speechText to Speech