Memory Augmented Lookup Dictionary based Language Modeling for Automatic Speech Recognition

2022-12-30 · Yukun Feng, Ming Tu, Rui Xia, Chuanzeng Huang, Yuxuan Wang

Recent studies have shown that using an external Language Model (LM) benefits the end-to-end Automatic Speech Recognition (ASR). However, predicting tokens that appear less frequently in the training set is still quite challenging. The long-tail prediction problems have been widely studied in many applications, but only been addressed by a few studies for ASR and LMs. In this paper, we propose a new memory augmented lookup dictionary based Transformer architecture for LM. The newly introduced lookup dictionary incorporates rich contextual information in training set, which is vital to correctly predict long-tail tokens. With intensive experiments on Chinese and English data sets, our proposed method is proved to outperform the baseline Transformer LM by a great margin on both word/character error rate and tail tokens error rate. This is achieved without impact on the decoding efficiency. Overall, we demonstrate the effectiveness of our proposed method in boosting the ASR decoding performance, especially for long-tail tokens.

📄 PDF Abstract BibTeX arXiv:2301.00066

Code (0)

등록된 구현이 없습니다.

Tasks

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language ModelingLanguage Modelling speech-recognition Speech Recognition

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Adam 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Residual Connection 설명 없음

Robust Dictionary Lookup in Multiple Noisy Orthographies

2017-04-01 · WS 2017 4 · Lingliang Zhang, Nizar Habash, Godfried Toussaint

We present the MultiScript Phonetic Search algorithm to address the problem of language learners looking up unfamiliar words that they heard. We apply it to Arabic dictionary lookup with noisy queries done using both the…

Transliteration

DictAS: A Framework for Class-Generalizable Few-Shot Anomaly Segmentation via Dictionary Lookup

2025-08-19 · Zhen Qu, Xian Tao, Xinyi Gong, ShiChen Qu 외 arxiv

Recent vision-language models (e.g., CLIP) have demonstrated remarkable class-generalizable ability to unseen classes in few-shot anomaly segmentation (FSAS), leveraging supervised prompt learning or fine-tuning on seen …

Self-Supervised Learning

Contextual Dictionary Lookup for Knowledge Graph Completion

2023-06-13 · Jining Wang, Delai Qiu, YouMing Liu, Yining Wang 외

Knowledge graph completion (KGC) aims to solve the incompleteness of knowledge graphs (KGs) by predicting missing links from known triples, numbers of knowledge graph embedding (KGE) models have been proposed to perform …

Graph Embedding Knowledge Graph Completion Knowledge Graph Embedding Knowledge Graphs+1

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

2026-01-12 · Xin Cheng, Wangding Zeng, Damai Dai, Qinyu Chen 외 arxiv

While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation. To address this…

LCNN: Lookup-based Convolutional Neural Network

2016-11-20 · CVPR 2017 7 · Hessam Bagherinezhad, Mohammad Rastegari, Ali Farhadi

Porting state of the art deep learning algorithms to resource constrained compute platforms (e.g. VR, AR, wearables) is extremely challenging. We propose a fast, compact, and accurate model for convolutional neural netwo…

Few-Shot Learning