Target Foresight Based Attention for Neural Machine Translation
In neural machine translation, an attention model is used to identify the aligned source words for a target word (target foresight word) in order to select translation context, but it does not make use of any information of this target foresight word at all. Previous work proposed an approach to improve the attention model by explicitly accessing this target foresight word and demonstrated the substantial gains in alignment task. However, this approach is useless in machine translation task on which the target foresight word is unavailable. In this paper, we propose a new attention model enhanced by the implicit information of target foresight word oriented to both alignment and translation tasks. Empirical experiments on Chinese-to-English and Japanese-to-English datasets show that the proposed attention model delivers significant improvements in terms of both alignment error rate and BLEU.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMachine TranslationTranslationSimilar Papers 제목 키워드 기반
Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation
The attentional mechanism has proven to be effective in improving end-to-end neural machine translation. However, due to the intricate structural divergence between natural languages, unidirectional attention-based model…
Machine TranslationTranslationWord AlignmentCLaD: Planning with Grounded Foresight via Cross-Modal Latent Dynamics
Robotic manipulation involves kinematic and semantic transitions that are inherently coupled via underlying actions. However, existing approaches plan within either semantic or latent space without explicitly aligning th…
Synchronous Syntactic Attention for Transformer Neural Machine Translation
This paper proposes a novel attention mechanism for Transformer Neural Machine Translation, {``}Synchronous Syntactic Attention,{''} inspired by synchronous dependency grammars. The mechanism synchronizes source-side and…
DecoderMachine TranslationTranslationLook-ahead Attention for Generation in Neural Machine Translation
The attention model has become a standard component in neural machine translation (NMT) and it guides translation process by selectively focusing on parts of the source sentence when predicting each target word. However,…
Machine TranslationNMTSentenceTranslationLook Harder: A Neural Machine Translation Model with Hard Attention
Soft-attention based Neural Machine Translation (NMT) models have achieved promising results on several translation tasks. These models attend all the words in the source sequence for each target token, which makes them …
Hard AttentionMachine TranslationNMTReinforcement Learning+2