Minimum Risk Training for Neural Machine Translation
We propose minimum risk training for end-to-end neural machine translation. Unlike conventional maximum likelihood estimation, minimum risk training is capable of optimizing model parameters directly with respect to arbitrary evaluation metrics, which are not necessarily differentiable. Experiments show that our approach achieves significant improvements over maximum likelihood estimation on a state-of-the-art neural machine translation system across various languages pairs. Transparent to architectures, our approach can be applied to more neural networks and potentially benefit more NLP tasks.
Code (1)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Lexicons and Minimum Risk Training for Neural Machine Translation: NAIST-CMU at WAT2016
This year, the Nara Institute of Science and Technology (NAIST)/Carnegie Mellon University (CMU) submission to the Japanese-English translation track of the 2016 Workshop on Asian Translation was based on attentional neu…
Machine TranslationNMTTranslationTHUMT: An Open Source Toolkit for Neural Machine Translation
This paper introduces THUMT, an open-source toolkit for neural machine translation (NMT) developed by the Natural Language Processing Group at Tsinghua University. THUMT implements the standard attention-based encoder-de…
DecoderMachine TranslationNMTTranslationUnderstanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation
Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words, and shows poor robustness to copy noise in training data or domain shift. Re…
Machine TranslationNMTTranslationIdentifying Weaknesses in Machine Translation Metrics Through Minimum Bayes Risk Decoding: A Case Study for COMET
Neural metrics have achieved impressive correlation with human judgements in the evaluation of machine translation systems, but before we can safely optimise towards such metrics, we should be aware of (and ideally elimi…
de-enMachine TranslationTranslationBLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Training
Automatic metrics play a crucial role in machine translation. Despite the widespread use of n-gram-based metrics, there has been a recent surge in the development of pre-trained model-based metrics that focus on measurin…
Machine TranslationSentenceTranslation