FUDGE: Controlled Text Generation With Future Discriminators
We propose Future Discriminators for Generation (FUDGE), a flexible and modular method for controlled text generation. Given a pre-existing model G for generating text from a distribution of interest, FUDGE enables conditioning on a desired attribute a (for example, formality) while requiring access only to G's output logits. FUDGE learns an attribute predictor operating on a partial sequence, and uses this predictor's outputs to adjust G's original probabilities. We show that FUDGE models terms corresponding to a Bayesian decomposition of the conditional distribution of G given attribute a. Moreover, FUDGE can easily compose predictors for multiple desired attributes. We evaluate FUDGE on three tasks -- couplet completion in poetry, topic control in language generation, and formality change in machine translation -- and observe gains in all three tasks.
Code (3)
Tasks
AttributeMachine TranslationText GenerationTranslationSimilar Papers 제목 키워드 기반
LLM-guided headline rewriting for clickability enhancement without clickbait
Enhancing reader engagement while preserving informational fidelity is a central challenge in controllable text generation for news media. Optimizing news headlines for reader engagement is often conflated with clickbait…
Text GenerationVisual FUDGE: Form Understanding via Dynamic Graph Editing
We address the problem of form understanding: finding text entities and the relationships/links between them in form images. The proposed FUDGE model formulates this problem on a graph of text elements (the vertices) and…
Entity LinkingFormToward Controlled Generation of Text
Generic generation and manipulation of text is challenging and has limited success compared to recent deep generative modeling in visual domain. This paper aims at generating plausible natural language sentences, whose a…
AttributeSentenceFudge: A light-weight solver for abstract argumentation based on SAT reductions
We present Fudge, an abstract argumentation solver that tightly integrates satisfiability solving technology to solve a series of abstract argumentation problems. While most of the encodings used by Fudge derive from sta…
Abstract ArgumentationTranslationTowards Automatic Evaluation of Task-Oriented Dialogue Flows
Task-oriented dialogue systems rely on predefined conversation schemes (dialogue flows) often represented as directed acyclic graphs. These flows can be manually designed or automatically generated from previously record…
Task-Oriented Dialogue Systems