On the True Distribution Approximation of Minimum Bayes-Risk Decoding
Minimum Bayes-risk (MBR) decoding has recently gained renewed attention in text generation. MBR decoding considers texts sampled from a model as pseudo-references and selects the text with the highest similarity to the others. Therefore, sampling is one of the key elements of MBR decoding, and previous studies reported that the performance varies by sampling methods. From a theoretical standpoint, this performance variation is likely tied to how closely the samples approximate the true distribution of references. However, this approximation has not been the subject of in-depth study. In this study, we propose using anomaly detection to measure the degree of approximation. We first closely examine the performance variation and then show that previous hypotheses about samples do not correlate well with the variation, but our introduced anomaly scores do. The results are the first to empirically support the link between the performance and the core assumption of MBR decoding.
Code (1)
Tasks
Anomaly DetectionText GenerationSimilar Papers 제목 키워드 기반
Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation
In NMT we search for the mode of the model distribution to form predictions. The mode and other high-probability translations found by beam search have been shown to often be inadequate in a number of ways. This prevents…
Machine TranslationNMTTranslation$α$-Variational Inference with Statistical Guarantees
We propose a family of variational approximations to Bayesian posterior distributions, called $\alpha$-VB, with provable statistical guarantees. The standard variational approximation is a special case of $\alpha$-VB wit…
Variational InferenceModel-Based Minimum Bayes Risk Decoding for Text Generation
Minimum Bayes Risk (MBR) decoding has been shown to be a powerful alternative to beam search decoding in a variety of text generation tasks. MBR decoding selects a hypothesis from a pool of hypotheses that has the least …
DecoderText GenerationVariational Bayes Inference in Digital Receivers
The digital telecommunications receiver is an important context for inference methodology, the key objective being to minimize the expected loss function in recovering the transmitted information. For that criterion, the…
LEMMAMoments and Root-Mean-Square Error of the Bayesian MMSE Estimator of Classification Error in the Gaussian Model
The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifi…
General Classification