GNCformer Enhanced Self-attention for Automatic Speech Recognition

2023-05-22 · J. Li, Z. Duan, S. Li, X. Yu, G. Yang

In this paper,an Enhanced Self-Attention (ESA) mechanism has been put forward for robust feature extraction.The proposed ESA is integrated with the recursive gated convolution and self-attention mechanism.In particular, the former is used to capture multi-order feature interaction and the latter is for global feature extraction.In addition, the location of interest that is suitable for inserting the ESA is also worth being explored.In this paper, the ESA is embedded into the encoder layer of the Transformer network for automatic speech recognition (ASR) tasks, and this newly proposed model is named GNCformer. The effectiveness of the GNCformer has been validated using two datasets, that are Aishell-1 and HKUST.Experimental results show that, compared with the Transformer network,0.8%CER,and 1.2%CER improvement for these two mentioned datasets, respectively, can be achieved.It is worth mentioning that only 1.4M additional parameters have been involved in our proposed GNCformer.

📄 PDF Abstract BibTeX arXiv:2305.12755

Code (0)

등록된 구현이 없습니다.

Tasks

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognition Speech Recognition

Methods 이 논문이 사용한 방법론

Attention 설명 없음

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Joint Training of Speech Enhancement and Self-supervised Model for Noise-robust ASR

2022-05-26 · Qiu-Shi Zhu, Jie Zhang, Zi-Qiang Zhang, Li-Rong Dai

Speech enhancement (SE) is usually required as a front end to improve the speech quality in noisy environments, while the enhanced speech might not be optimal for automatic speech recognition (ASR) systems due to speech …

Automatic Speech Recognition Automatic Speech Recognition (ASR)Speech Enhancement speech-recognition+1

Cross-attention conformer for context modeling in speech enhancement for ASR

2021-10-30 · Arun Narayanan, Chung-Cheng Chiu, Tom O'Malley, Quan Wang 외

This work introduces \emph{cross-attention conformer}, an attention-based architecture for context modeling in speech enhancement. Given that the context information can often be sequential, and of different length as th…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Speech Enhancement speech-recognition+1

E-Branchformer: Branchformer with Enhanced merging for speech recognition

2022-09-30 · Kwangyoun Kim, Felix Wu, Yifan Peng, Jing Pan 외

Conformer, combining convolution and self-attention sequentially to capture both local and global information, has shown remarkable performance and is currently regarded as the state-of-the-art for automatic speech recog…

Automatic Speech Recognition Automatic Speech Recognition (ASR)speech-recognition Speech Recognition

Adaptive Sparse and Monotonic Attention for Transformer-based Automatic Speech Recognition

2022-09-30 · Chendong Zhao, Jianzong Wang, Wen qi Wei, Xiaoyang Qu 외

The Transformer architecture model, based on self-attention and multi-head attention, has achieved remarkable success in offline end-to-end Automatic Speech Recognition (ASR). However, self-attention and multi-head atten…

Automatic Speech Recognition Automatic Speech Recognition (ASR)speech-recognition Speech Recognition

End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation

2022-04-01 · Xuankai Chang, Takashi Maekaku, Yuya Fujita, Shinji Watanabe

This work presents our end-to-end (E2E) automatic speech recognition (ASR) model targetting at robust speech recognition, called Integraded speech Recognition with enhanced speech Input for Self-supervised learning repre…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Robust Speech Recognition Self-Supervised Learning+3