Controlling Summarization Length Through EOS Token Weighting
Controlling the length of generated text can be crucial in various text-generation tasks, including summarization. Existing methods often require complex model alterations, limiting compatibility with pre-trained models. We address these limitations by developing a simple approach for controlling the length of automatic text summaries by increasing the importance of correctly predicting the EOS token in the cross-entropy loss computation. The proposed methodology is agnostic to architecture and decoding algorithms and orthogonal to other inference-time techniques to control the generation length. We tested it with encoder-decoder and modern GPT-style LLMs, and show that this method can control generation length, often without affecting the quality of the summary.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderText GenerationSimilar Papers 제목 키워드 기반
Summarization with Precise Length Control
Many applications of text generation such as summarization benefit from accurately controlling the text length. Existing approaches on length-controlled summarization either result in degraded performance or can only con…
Text GenerationPrecise Length Control in Large Language Models
Large Language Models (LLMs) are increasingly used in production systems, powering applications such as chatbots, summarization, and question answering. Despite their success, controlling the length of their response rem…
DecoderDocument SummarizationQuestion AnsweringPrompt-Based One-Shot Exact Length-Controlled Generation with LLMs
Controlling the length of text produced by large language models (LLMs) remains challenging: models frequently overshoot or undershoot explicit length instructions because they cannot reliably keep an internal token coun…
Instruction FollowingPrompt EngineeringLogit Reweighting for Topic-Focused Summarization
Generating abstractive summaries that adhere to a specific topic remains a significant challenge for language models. While standard approaches, such as fine-tuning, are resource-intensive, simpler methods like prompt en…
Prompt EngineeringLenAtten: An Effective Length Controlling Unit For Text Summarization
Fixed length summarization aims at generating summaries with a preset number of words or characters. Most recent researches incorporate length information with word embeddings as the input to the recurrent decoding unit,…
Text SummarizationWord Embeddings