Learning When to Attend for Neural Machine Translation
In the past few years, attention mechanisms have become an indispensable component of end-to-end neural machine translation models. However, previous attention models always refer to some source words when predicting a target word, which contradicts with the fact that some target words have no corresponding source words. Motivated by this observation, we propose a novel attention model that has the capability of determining when a decoder should attend to source words and when it should not. Experimental results on NIST Chinese-English translation tasks show that the new model achieves an improvement of 0.8 BLEU score over a state-of-the-art baseline.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMachine TranslationTranslationSimilar Papers 제목 키워드 기반
Extract and Attend: Improving Entity Translation in Neural Machine Translation
While Neural Machine Translation(NMT) has achieved great progress in recent years, it still suffers from inaccurate translation of entities (e.g., person/organization name, location), due to the lack of entity training i…
DecoderMachine TranslationNMTSentence+1Learning Hard Retrieval Decoder Attention for Transformers
The Transformer translation model is based on the multi-head attention mechanism, which can be parallelized easily. The multi-head attention network performs the scaled dot-product attention function in parallel, empower…
DecoderMachine TranslationRetrievalSentence+1Neural Machine Translation with Recurrent Attention Modeling
Knowing which words have been attended to in previous time steps while generating a translation is a rich source of information for predicting what words will be attended to in the future. We improve upon the attention m…
Machine TranslationTranslationShow, Attend and Translate: Unpaired Multi-Domain Image-to-Image Translation with Visual Attention
Recently unpaired multi-domain image-to-image translation has attracted great interests and obtained remarkable progress, where a label vector is utilized to indicate multi-domain information. In this paper, we propose S…
AttributeGenerative Adversarial NetworkImage-to-Image TranslationTranslationPlan, Attend, Generate: Character-level Neural Machine Translation with Planning in the Decoder
We investigate the integration of a planning mechanism into an encoder-decoder architecture with an explicit alignment for character-level machine translation. We develop a model that plans ahead when it computes alignme…
DecoderMachine TranslationTranslation