Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation
Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words, and shows poor robustness to copy noise in training data or domain shift. Recent work has tied these shortcomings to beam search -- the de facto standard inference algorithm in NMT -- and Eikema & Aziz (2020) propose to use Minimum Bayes Risk (MBR) decoding on unbiased samples instead. In this paper, we empirically investigate the properties of MBR decoding on a number of previously reported biases and failure cases of beam search. We find that MBR still exhibits a length and token frequency bias, owing to the MT metrics used as utility functions, but that MBR also increases robustness against copy noise in the training data and domain shift.
Code (1)
Tasks
Machine TranslationNMTTranslationSimilar Papers 제목 키워드 기반
Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation
For extended periods of time, sequence generation models rely on beam search algorithm to generate output sequence. However, the correctness of beam search degrades when the a model is over-confident about a suboptimal p…
GPUMachine TranslationRerankingTranslationDocument-Level Text Generation with Minimum Bayes Risk Decoding using Optimal Transport
Document-level text generation tasks are known to be more difficult than sentence-level text generation tasks as they require the understanding of longer context to generate high-quality texts. In this paper, we investig…
Document Level Machine TranslationImage CaptioningMachine TranslationSentence+2Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding
Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive. We show how the recently developed Re…
Machine TranslationNMTreinforcement-learningReinforcement Learning+2Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation
Maximum a posteriori decoding, a commonly used method for neural machine translation (NMT), aims to maximize the estimated posterior probability. However, high estimated probability does not always lead to high translati…
Machine TranslationNMTRerankingTranslationCentroid-Based Efficient Minimum Bayes Risk Decoding
Minimum Bayes risk (MBR) decoding achieved state-of-the-art translation performance by using COMET, a neural metric that has a high correlation with human evaluation. However, MBR decoding requires quadratic time since i…
de-enTranslation