Convolutional Sequence to Sequence Learning
The prevalent approach to sequence to sequence learning maps an input sequence to a variable length output sequence via recurrent neural networks. We introduce an architecture based entirely on convolutional neural networks. Compared to recurrent models, computations over all elements can be fully parallelized during training and optimization is easier since the number of non-linearities is fixed and independent of the input length. Our use of gated linear units eases gradient propagation and we equip each decoder layer with a separate attention module. We outperform the accuracy of the deep LSTM setup of Wu et al. (2016) on both WMT'14 English-German and WMT'14 English-French translation at an order of magnitude faster speed, both on GPU and CPU.
Code (37)
Tasks
Bangla Spelling Error CorrectionCPUDecoderGPUImage ClassificationMachine TranslationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Convolutional Sequence to Sequence Non-intrusive Load Monitoring
A convolutional sequence to sequence non-intrusive load monitoring model is proposed in this paper. Gated linear unit convolutional layers are used to extract information from the sequences of aggregate electricity consu…
Non-Intrusive Load MonitoringAn Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent networks on tasks such as audio synthes…
Audio SynthesisLanguage ModellingMachine TranslationMusic Modeling+3A Context-aware Convolutional Natural Language Generation model for Dialogue Systems
Natural language generation (NLG) is an important component in spoken dialog systems (SDSs). A model for NLG involves sequence to sequence learning. State-of-the-art NLG models are built using recurrent neural network (R…
Machine TranslationSemantic correspondenceSpoken Dialogue SystemsText Generation+1Comparing Recurrent and Convolutional Architectures for English-Hindi Neural Machine Translation
In this paper, we empirically compare the two encoder-decoder neural machine translation architectures: convolutional sequence to sequence model (ConvS2S) and recurrent sequence to sequence model (RNNS2S) for English-Hin…
DecoderImage CaptioningLanguage ModelingLanguage Modelling+4DNA-GCN: Graph convolutional networks for predicting DNA-protein binding
Predicting DNA-protein binding is an important and classic problem in bioinformatics. Convolutional neural networks have outperformed conventional methods in modeling the sequence specificity of DNA-protein binding. Howe…
Specificity