Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding
One of the most important challenges in text generation systems is to produce outputs that are not only correct but also diverse. Recently, Minimum Bayes-Risk (MBR) decoding has gained prominence for generating sentences of the highest quality among the decoding algorithms. However, existing algorithms proposed for generating diverse outputs are predominantly based on beam search or random sampling, thus their output quality is capped by these underlying methods. In this paper, we investigate an alternative approach -- we develop diversity-promoting decoding algorithms by enforcing diversity objectives to MBR decoding. We propose two variants of MBR, Diverse MBR (DMBR) and $k$-medoids MBR (KMBR), methods to generate a set of sentences with high quality and diversity. We evaluate DMBR and KMBR on a variety of directed text generation tasks using encoder-decoder models and a large language model with prompting. The experimental results show that the proposed method achieves a better trade-off than the diverse beam search and sampling algorithms.
Code (1)
Tasks
DecoderDiversityLanguage ModelingLanguage ModellingLarge Language ModelText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Cascading CMA-ES Instances for Generating Input-diverse Solution Batches
Rather than obtaining a single good solution for a given optimization problem, users often seek alternative design choices, because the best-found solution may perform poorly with respect to additional objectives or cons…
DiversityNecessary and Sufficient Watermark for Large Language Models
In recent years, large language models (LLMs) have achieved remarkable performances in various NLP tasks. They can generate texts that are indistinguishable from those written by humans. Such remarkable performance of LL…
ArticlesMachine TranslationSummarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries
The difficulty of generating coherent long texts lies in the fact that existing models overwhelmingly focus on predicting local words, and cannot make high level plans on what to generate or capture the high-level discou…
Text GenerationExploring Precision and Recall to assess the quality and diversity of LLMs
We introduce a novel evaluation framework for Large Language Models (LLMs) such as \textsc{Llama-2} and \textsc{Mistral}, focusing on importing Precision and Recall metrics from image generation to text generation. This …
DiversityImage GenerationText GenerationDiverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors
Generating high quality texts with high diversity is important for many NLG applications, but current methods mostly focus on building deterministic models to generate higher quality texts and do not provide many options…
DecoderDiversityGaussian ProcessesParaphrase Generation+5