Comparison of the effects of attention mechanism on translation tasks of different lengths of ambiguous words
In recent years, attention mechanism has been widely used in various neural machine translation tasks based on encoder decoder. This paper focuses on the performance of encoder decoder attention mechanism in word sense disambiguation task with different text length, trying to find out the influence of context marker on attention mechanism in word sense disambiguation task. We hypothesize that attention mechanisms have similar performance when translating texts of different lengths. Our conclusion is that the alignment effect of attention mechanism is magnified in short text translation tasks with ambiguous nouns, while the effect of attention mechanism is far less than expected in long-text tasks, which means that attention mechanism is not the main mechanism for NMT model to feed WSD to integrate context information. This may mean that attention mechanism pays more attention to ambiguous nouns than context markers. The experimental results show that with the increase of text length, the performance of NMT model using attention mechanism will gradually decline.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMachine TranslationNMTTranslationWord Sense DisambiguationSimilar Papers 제목 키워드 기반
Double Attention-based Multimodal Neural Machine Translation with Semantic Image Regions
Existing studies on multimodal neural machine translation (MNMT) have mainly focused on the effect of combining visual and textual modalities to improve translations. However, it has been suggested that the visual modali…
Machine TranslationTranslationEfficient Attention using a Fixed-Size Memory Representation
The standard content-based attention mechanism typically used in sequence-to-sequence models is computationally expensive as it requires the comparison of large encoder and decoder states at each time step. In this work,…
DecoderTranslationSPA-GAN: Spatial Attention GAN for Image-to-Image Translation
Image-to-image translation is to learn a mapping between images from a source domain and images from a target domain. In this paper, we introduce the attention mechanism directly to the generative adversarial network (GA…
Generative Adversarial NetworkImage-to-Image TranslationSingle Particle AnalysisTranslationTraining Deeper Neural Machine Translation Models with Transparent Attention
While current state-of-the-art NMT models, such as RNN seq2seq and Transformers, possess a large number of parameters, they are still shallow in comparison to convolutional models used for both text and vision applicatio…
Machine TranslationNMTTranslationAn Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation
Recent work has shown that the encoder-decoder attention mechanisms in neural machine translation (NMT) are different from the word alignment in statistical machine translation. In this paper, we focus on analyzing encod…
DecoderMachine TranslationNMTTranslation+2