Fine-Grained Attention Mechanism for Neural Machine Translation
Neural machine translation (NMT) has been a new paradigm in machine translation, and the attention mechanism has become the dominant approach with the state-of-the-art records in many language pairs. While there are variants of the attention mechanism, all of them use only temporal attention where one scalar value is assigned to one context vector corresponding to a source word. In this paper, we propose a fine-grained (or 2D) attention mechanism where each dimension of a context vector will receive a separate attention score. In experiments with the task of En-De and En-Fi translation, the fine-grained attention method improves the translation quality in terms of BLEU score. In addition, our alignment analysis reveals how the fine-grained attention mechanism exploits the internal structure of context vectors.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTTranslationSimilar Papers 제목 키워드 기반
Lost in the Source Language: How Large Language Models Evaluate the Quality of Machine Translation
This study investigates how Large Language Models (LLMs) leverage source and reference data in machine translation evaluation task, aiming to better understand the mechanisms behind their remarkable performance in this t…
Machine TranslationTranslationFine Grained Human Evaluation for English-to-Chinese Machine Translation: A Case Study on Scientific Text
Recent research suggests that neural machine translation (MT) in the news domain has reached human-level performance, but for other professional domains, it is far below the level. In this paper, we conduct a fine-graine…
Machine TranslationNMTTranslationImproving Attention Modeling with Implicit Distortion and Fertility for Machine Translation
In neural machine translation, the attention mechanism facilitates the translation process by producing a soft alignment between the source sentence and the target sentence. However, without dedicated distortion and fert…
DecoderMachine TranslationSentenceTranslationAttention based Sequence to Sequence Learning for Machine Translation of Low Resourced Indic Languages -- A case of Sanskrit to Hindi
Deep Learning techniques are powerful in mimicking humans in a particular set of problems. They have achieved a remarkable performance in complex learning tasks. Deep learning inspired Neural Machine Translation (NMT) is…
Machine TranslationNMTTranslationName Translation based on Fine-grained Named Entity Recognition in a Single Language
We propose named entity abstraction methods with fine-grained named entity labels for improving statistical machine translation (SMT). The methods are based on a bilingual named entity recognizer that uses a monolingual …
Machine Translationnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2