Convolutional Attention-based Seq2Seq Neural Network for End-to-End ASR

2017-10-12 · Dan Lim

This thesis introduces the sequence to sequence model with Luong's attention mechanism for end-to-end ASR. It also describes various neural network algorithms including Batch normalization, Dropout and Residual network which constitute the convolutional attention-based seq2seq neural network. Finally the proposed model proved its effectiveness for speech recognition achieving 15.8% phoneme error rate on TIMIT dataset.

📄 PDF Abstract BibTeX arXiv:1710.04515

Code (0)

등록된 구현이 없습니다.

Tasks

speech-recognition Speech Recognition

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Seq2Seq Seq2Seq, or Sequence To Sequence, is a model used in sequence prediction tasks, such as language modelling and machine translation. The idea is to use one…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

RFAConv: Innovating Spatial Attention and Standard Convolutional Operation

2023-04-06 · Xin Zhang, Chen Liu, Degang Yang, Tingting Song 외

Spatial attention has been widely used to improve the performance of convolutional neural networks. However, it has certain limitations. In this paper, we propose a new perspective on the effectiveness of spatial attenti…

Classification Object Detection Semantic Segmentation

FocusNet: An attention-based Fully Convolutional Network for Medical Image Segmentation

2019-02-08 · Chaitanya Kaul, Suresh Manandhar, Nick Pears

We propose a novel technique to incorporate attention within convolutional neural networks using feature maps generated by a separate convolutional autoencoder. Our attention architecture is well suited for incorporation…

Image Segmentation Lesion Segmentation Medical Image Segmentation Segmentation+2

On the Relationship between Self-Attention and Convolutional Layers

2019-11-08 · ICLR 2020 1 · Jean-Baptiste Cordonnier, Andreas Loukas, Martin Jaggi

Recent trends of incorporating attention mechanisms in vision have led researchers to reconsider the supremacy of convolutional layers as a primary building block. Beyond helping CNNs to handle long-range dependencies, R…

Image Classification

EAANet: Efficient Attention Augmented Convolutional Networks

2022-06-03 · Runqing Zhang, Tianshu Zhu

Humans can effectively find salient regions in complex scenes. Self-attention mechanisms were introduced into Computer Vision (CV) to achieve this. Attention Augmented Convolutional Network (AANet) is a mixture of convol…

Self-Attention Generative Adversarial Network for Speech Enhancement

2020-10-18 · Huy Phan, Huy Le Nguyen, Oliver Y. Chén, Philipp Koch 외

Existing generative adversarial networks (GANs) for speech enhancement solely rely on the convolution operation, which may obscure temporal dependencies across the sequence input. To remedy this issue, we propose a self-…

Generative Adversarial Network Speech Enhancement