EEG based Continuous Speech Recognition using Transformers

2019-12-31 · Gautam Krishna, Co Tran, Mason Carnahan, Ahmed H. Tewfik

In this paper we investigate continuous speech recognition using electroencephalography (EEG) features using recently introduced end-to-end transformer based automatic speech recognition (ASR) model. Our results demonstrate that transformer based model demonstrate faster training compared to recurrent neural network (RNN) based sequence-to-sequence EEG models and better performance during inference time for smaller test set vocabulary but as we increase the vocabulary size, the performance of the RNN based models were better than transformer based model on a limited English vocabulary.

📄 PDF Abstract BibTeX arXiv:2001.00501

Code (0)

등록된 구현이 없습니다.

Tasks

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)EEGElectroencephalogram (EEG)speech-recognition Speech Recognition

Methods 이 논문이 사용한 방법론

Test 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Residual Connection 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Weak-Attention Suppression For Transformer Based Speech Recognition

2020-05-18 · Yangyang Shi, Yongqiang Wang, Chunyang Wu, Christian Fuegen 외

Transformers, originally proposed for natural language processing (NLP) tasks, have recently achieved great success in automatic speech recognition (ASR). However, adjacent acoustic units (i.e., frames) are highly correl…

Automatic Speech Recognition Automatic Speech Recognition (ASR)speech-recognition Speech Recognition

Automatic Speech Recognition with BERT and CTC Transformers: A Review

2024-10-12 · Noussaiba Djeffal, Hamza Kheddar, Djamel Addou, Ahmed Cherif Mazari 외

This review paper provides a comprehensive analysis of recent advances in automatic speech recognition (ASR) with bidirectional encoder representations from transformers BERT and connectionist temporal classification (CT…

Automatic Speech Recognition Automatic Speech Recognition (ASR)speech-recognition Speech Recognition

Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis

2024-07-13 · Xilin Jiang, Yinghao Aaron Li, Adrian Nicolas Florea, Cong Han 외

It is too early to conclude that Mamba is a better alternative to transformers for speech before comparing Mamba with transformers in terms of both performance and efficiency in multiple speech-related tasks. To reach th…

Mamba speech-recognition Speech Recognition Speech Separation+1

Continuous Speech Recognition using EEG and Video

2019-12-16 · Gautam Krishna, Mason Carnahan, Co Tran, Ahmed H. Tewfik

In this paper we investigate whether electroencephalography (EEG) features can be used to improve the performance of continuous visual speech recognition systems. We implemented a connectionist temporal classification (C…

Automatic Speech Recognition Automatic Speech Recognition (ASR)EEG Electroencephalogram (EEG)+4

Speech Recognition With No Speech Or With Noisy Speech Beyond English

2019-06-17 · Gautam Krishna, Co Tran, Yan Han, Mason Carnahan 외

In this paper we demonstrate continuous noisy speech recognition using connectionist temporal classification (CTC) model on limited Chinese vocabulary using electroencephalography (EEG) features with no speech signal as …

EEG Electroencephalogram (EEG)General Classification Noisy Speech Recognition+2