홈 › Papers

Multi-Head State Space Model for Speech Recognition

2023-05-21 · Yassir Fathullah, Chunyang Wu, Yuan Shangguan, Junteng Jia, Wenhan Xiong, Jay Mahadeokar, Chunxi Liu, Yangyang Shi, Ozlem Kalinli, Mike Seltzer, Mark J. F. Gales

State space models (SSMs) have recently shown promising results on small-scale sequence and language modelling tasks, rivalling and outperforming many attention-based approaches. In this paper, we propose a multi-head state space (MH-SSM) architecture equipped with special gating mechanisms, where parallel heads are taught to learn local and global temporal dynamics on sequence data. As a drop-in replacement for multi-head attention in transformer encoders, this new model significantly outperforms the transformer transducer on the LibriSpeech speech recognition corpus. Furthermore, we augment the transformer block with MH-SSMs layers, referred to as the Stateformer, achieving state-of-the-art performance on the LibriSpeech task, with word error rates of 1.76\%/4.37\% on the development and 1.91\%/4.36\% on the test sets without using an external language model.

📄 PDF Abstract BibTeX arXiv:2305.12498

Code (0)

등록된 구현이 없습니다.

Tasks

Language ModelingLanguage Modellingmodelspeech-recognition Speech RecognitionState Space Models

Methods 이 논문이 사용한 방법론

Test 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Pyramid Multi-branch Fusion DCNN with Multi-Head Self-Attention for Mandarin Speech Recognition

2023-03-23 · Kai Liu, Hailiang Xiong, Gangqiang Yang, Zhengfeng Du 외

As one of the major branches of automatic speech recognition, attention-based models greatly improves the feature representation ability of the model. In particular, the multi-head mechanism is employed in the attention,…

Automatic Speech Recognition speech-recognition Speech Recognition

Direction-Aware Joint Adaptation of Neural Speech Enhancement and Recognition in Real Multiparty Conversational Environments

2022-07-15 · Yicheng Du, Aditya Arie Nugraha, Kouhei Sekiguchi, Yoshiaki Bando 외

This paper describes noisy speech recognition for an augmented reality headset that helps verbal communication within real multiparty conversational environments. A major approach that has actively been studied in simula…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Distant Speech Recognition Noisy Speech Recognition+3

Improving Speech Emotion Recognition Through Focus and Calibration Attention Mechanisms

2022-08-21 · Junghun Kim, Yoojin An, Jihie Kim

Attention has become one of the most commonly used mechanisms in deep learning approaches. The attention mechanism can help the system focus more on the feature space's critical regions. For example, high amplitude regio…

Emotion Recognition Speech Emotion Recognition

Adaptive Sparse and Monotonic Attention for Transformer-based Automatic Speech Recognition

2022-09-30 · Chendong Zhao, Jianzong Wang, Wen qi Wei, Xiaoyang Qu 외

The Transformer architecture model, based on self-attention and multi-head attention, has achieved remarkable success in offline end-to-end Automatic Speech Recognition (ASR). However, self-attention and multi-head atten…

Automatic Speech Recognition Automatic Speech Recognition (ASR)speech-recognition Speech Recognition

The History of Speech Recognition to the Year 2030

2021-07-30 · Awni Hannun

The decade from 2010 to 2020 saw remarkable improvements in automatic speech recognition. Many people now use speech recognition on a daily basis, for example to perform voice search queries, send text messages, and inte…