Context-Aware Smoothing for Neural Machine Translation
In Neural Machine Translation (NMT), each word is represented as a low-dimension, real-value vector for encoding its syntax and semantic information. This means that even if the word is in a different sentence context, it is represented as the fixed vector to learn source representation. Moreover, a large number of Out-Of-Vocabulary (OOV) words, which have different syntax and semantic information, are represented as the same vector representation of {``}unk{''}. To alleviate this problem, we propose a novel context-aware smoothing method to dynamically learn a sentence-specific vector for each word (including OOV words) depending on its local context words in a sentence. The learned context-aware representation is integrated into the NMT to improve the translation performance. Empirical results on NIST Chinese-to-English translation task show that the proposed approach achieves 1.78 BLEU improvements on average over a strong attentional NMT, and outperforms some existing systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTRepresentation LearningSentenceTranslationSimilar Papers 제목 키워드 기반
Unifying Input and Output Smoothing in Neural Machine Translation
Soft contextualized data augmentation is a recent method that replaces one-hot representation of words with soft posterior distributions of an external language model, smoothing the input of neural machine translation sy…
Data AugmentationLanguage ModelingLanguage ModellingMachine Translation+1Towards a Better Understanding of Label Smoothing in Neural Machine Translation
In order to combat overfitting and in pursuit of better generalization, label smoothing is widely applied in modern neural machine translation systems. The core idea is to penalize over-confident outputs and regularize t…
Machine TranslationTranslationGloss2Text: Sign Language Gloss translation using LLMs and Semantically Aware Label Smoothing
Sign language translation from video to spoken text presents unique challenges owing to the distinct grammar, expression nuances, and high variation of visual appearance across different speakers and contexts. The interm…
Data AugmentationSign Language TranslationTranslationWhy Neural Machine Translation Prefers Empty Outputs
We investigate why neural machine translation (NMT) systems assign high probability to empty translations. We find two explanations. First, label smoothing makes correct-length translations less confident, making it easi…
Machine TranslationNMTTranslationFocus on the Target’s Vocabulary: Masked Label Smoothing for Machine Translation
Label smoothing and vocabulary sharing are two widely used techniques in neural machine translation models. However, we argue that jointly adopting these two techniques can be conflicting and even leads to sub-optimal pe…
Machine TranslationTranslation