RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition
We compare the fast training and decoding speed of RETURNN of attention models for translation, due to fast CUDA LSTM kernels, and a fast pure TensorFlow beam search decoder. We show that a layer-wise pretraining scheme for recurrent attention models gives over 1% BLEU improvement absolute and it allows to train deeper recurrent encoder networks. Promising preliminary results on max. expected BLEU training are presented. We are able to train state-of-the-art models for translation and end-to-end models for speech recognition and show results on WMT 2017 and Switchboard. The flexibility of RETURNN allows a fast research feedback loop to experiment with alternative architectures, and its generality allows to use it on a wide range of applications.
Code (3)
Tasks
Decoderspeech-recognitionSpeech RecognitionTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
NMT-Keras: a Very Flexible Toolkit with a Focus on Interactive NMT and Online Learning
We present NMT-Keras, a flexible toolkit for training deep learning models, which puts a particular emphasis on the development of advanced applications of neural machine translation systems, such as interactive-predicti…
General ClassificationMachine TranslationNMTQuestion Answering+6NMTPY: A Flexible Toolkit for Advanced Neural Machine Translation Systems
In this paper, we present nmtpy, a flexible Python toolkit based on Theano for training Neural Machine Translation and other neural sequence-to-sequence architectures. nmtpy decouples the specification of a network from …
Machine TranslationMultimodal Machine TranslationTranslationTexar: A Modularized, Versatile, and Extensible Toolbox for Text Generation
We introduce Texar, an open-source toolkit aiming to support the broad set of text generation tasks. Different from many existing toolkits that are specialized for specific applications (e.g., neural machine translation)…
Image CaptioningMachine TranslationText GenerationText Summarization+1Flexible retrieval with NMSLIB and FlexNeuART
Our objective is to introduce to the NLP community an existing k-NN search library NMSLIB, a new retrieval toolkit FlexNeuART, as well as their integration capabilities. NMSLIB, while being one the fastest k-NN search li…
Re-RankingRetrievalRETURNN: The RWTH Extensible Training framework for Universal Recurrent Neural Networks
In this work we release our extensible and easily configurable neural network training software. It provides a rich set of functional layers with a particular focus on efficient training of recurrent neural network topol…