Adaptable End-to-End ASR Models using Replaceable Internal LMs and Residual Softmax

2023-02-16 · Keqi Deng, Philip C. Woodland

End-to-end (E2E) automatic speech recognition (ASR) implicitly learns the token sequence distribution of paired audio-transcript training data. However, it still suffers from domain shifts from training to testing, and domain adaptation is still challenging. To alleviate this problem, this paper designs a replaceable internal language model (RILM) method, which makes it feasible to directly replace the internal language model (LM) of E2E ASR models with a target-domain LM in the decoding stage when a domain shift is encountered. Furthermore, this paper proposes a residual softmax (R-softmax) that is designed for CTC-based E2E ASR models to adapt to the target domain without re-training during inference. For E2E ASR models trained on the LibriSpeech corpus, experiments showed that the proposed methods gave a 2.6% absolute WER reduction on the Switchboard data and a 1.0% WER reduction on the AESRC2020 corpus while maintaining intra-domain ASR results.

📄 PDF Abstract BibTeX arXiv:2302.08579

Code (0)

등록된 구현이 없습니다.

Tasks

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Domain AdaptationLanguage ModelingLanguage Modelling speech-recognition Speech Recognition

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Neural Video Fields Editing

2023-12-12 · Shuzhou Yang, Chong Mou, Jiwen Yu, YuHan Wang 외

Diffusion models have revolutionized text-driven video editing. However, applying these methods to real-world editing encounters two significant challenges: (1) the rapid increase in GPU memory demand as the number of fr…

GPU Video Editing

Residual Language Model for End-to-end Speech Recognition

2022-06-15 · Emiru Tsunoo, Yosuke Kashiwagi, Chaitanya Narisetty, Shinji Watanabe

End-to-end automatic speech recognition suffers from adaptation to unknown target domain speech despite being trained with a large amount of paired audio--text data. Recent studies estimate a linguistic bias of the model…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Domain Adaptation Language Modeling+3

SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation

2026-07-23 · Junsong Chen, Jincheng Yu, Yitong Li, Shuchen Xue 외 arxiv

We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches…

Video Generation

Attention Is Where You Attack

2026-04-30 · Aviral Srivastava, Sourav Panda arxiv

Safety-aligned large language models rely on RLHF and instruction tuning to refuse harmful requests, yet the internal mechanisms implementing safety behavior remain poorly understood. We introduce the Attention Redistrib…

Adversarial Attack

Decoupled Structure for Improved Adaptability of End-to-End Models

2023-08-25 · Keqi Deng, Philip C. Woodland

Although end-to-end (E2E) trainable automatic speech recognition (ASR) has shown great success by jointly learning acoustic and linguistic information, it still suffers from the effect of domain shifts, thus limiting pot…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Decoder Domain Adaptation+3