Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model
Neural machine translation has shown very promising results lately. Most NMT models follow the encoder-decoder framework. To make encoder-decoder models more flexible, attention mechanism was introduced to machine translation and also other tasks like speech recognition and image captioning. We observe that the quality of translation by attention-based encoder-decoder can be significantly damaged when the alignment is incorrect. We attribute these problems to the lack of distortion and fertility models. Aiming to resolve these problems, we propose new variations of attention-based encoder-decoder and compare them with other models on machine translation. Our proposed method achieved an improvement of 2 BLEU points over the original attention-based encoder-decoder.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeDecoderImage CaptioningMachine TranslationNMTspeech-recognitionSpeech RecognitionTranslationSimilar Papers 제목 키워드 기반
Improving Attention Modeling with Implicit Distortion and Fertility for Machine Translation
In neural machine translation, the attention mechanism facilitates the translation process by producing a soft alignment between the source sentence and the target sentence. However, without dedicated distortion and fert…
DecoderMachine TranslationSentenceTranslationIncorporating Structural Alignment Biases into an Attentional Neural Translation Model
Neural encoder-decoder models of machine translation have achieved impressive results, rivalling traditional translation models. However their modelling formulation is overly simplistic, and omits several key inductive b…
DecoderMachine TranslationTranslationNon-local Attention Optimized Deep Image Compression
This paper proposes a novel Non-Local Attention Optimized Deep Image Compression (NLAIC) framework, which is built on top of the popular variational auto-encoder (VAE) structure. Our NLAIC framework embeds non-local oper…
Image CompressionMS-SSIMSSIMNeural Machine Translation with Recurrent Attention Modeling
Knowing which words have been attended to in previous time steps while generating a translation is a rich source of information for predicting what words will be attended to in the future. We improve upon the attention m…
Machine TranslationTranslationChar-Net: A Character-Aware Neural Network for Distorted Scene Text Recognition
In this paper, we present a Character-Aware Neural Network (Char-Net) for recognizing distorted scene text. Our CharNet is composed of a word-level encoder, a character-level encoder, and a LSTM-based decoder. Unlike p…
DecoderScene Text Recognition