A Frustratingly Simple Decoding Method for Neural Text Generation
We introduce a frustratingly simple, super efficient and surprisingly effective decoding method, which we call Frustratingly Simple Decoding (FSD), for neural text generation. The idea behind FSD is straightforward: we build an anti-LM based on previously generated text and use this anti-LM to penalize future generation of what has been generated. The anti-LM can be implemented as simple as an n-gram language model or a vectorized variant. In this way, FSD introduces no extra model parameters and negligible computational overhead (FSD can be as fast as greedy search). Despite the simplicity, FSD is surprisingly effective; Experiments show that FSD can outperform the canonical methods to date (i.e., nucleus sampling) as well as several strong baselines that were proposed recently.
Code (1)
Tasks
Language ModelingLanguage ModellingText GenerationSimilar Papers 제목 키워드 기반
Frustratingly Easy Model Ensemble for Abstractive Summarization
Ensemble methods, which combine multiple models at decoding time, are now widely known to be effective for text-generation tasks. However, they generally increase computational costs, and thus, there have been many studi…
Abstractive Text SummarizationDensity EstimationHeadline Generationmodel+3Frustratingly Simple Few-Shot Slot Tagging
Frustratingly Simple Prompting-based Text Denoising
This paper introduces a novel perspective on the automated essay scoring (AES) task, challenging the conventional view of the ASAP dataset as a static entity. Employing simple text denoising techniques using prompting, w…
Automated Essay ScoringDenoisingregressionA Frustratingly Simple Approach for End-to-End Image Captioning
Image Captioning is a fundamental task to join vision and language, concerning about cross-modal understanding and text generation. Recent years witness the emerging attention on image captioning. Most of existing works …
DecoderImage CaptioningObjectText GenerationFrustratingly Easy Uncertainty Estimation for Distribution Shift
Distribution shift is an important concern in deep image classification, produced either by corruption of the source images, or a complete change, with the solution involving domain adaptation. While the primary goal is …
Domain Adaptationimage-classificationImage ClassificationUnsupervised Domain Adaptation