Posterior Control of Blackbox Generation
Text generation often requires high-precision output that obeys task-specific rules. This fine-grained control is difficult to enforce with off-the-shelf deep learning models. In this work, we consider augmenting neural generation models with discrete control states learned through a structured latent-variable approach. Under this formulation, task-specific knowledge can be encoded through a range of rich, posterior constraints that are effectively trained into the model. This approach allows users to ground internal model decisions based on prior knowledge, without sacrificing the representational power of neural generative models. Experiments consider applications of this approach for text generation. We find that this method improves over standard benchmarks, while also providing fine-grained control.
Code (2)
Tasks
Text GenerationSimilar Papers 제목 키워드 기반
Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models
Due to the unidirectional nature of prevalent autoregressive generation models, recent work on controlled generation based on global text attributes has either required attribute-based fine-tuning of the base language mo…
AttributeLanguage ModelingLanguage ModellingStyle Transfer+1Sketch-Guided Constrained Decoding for Boosting Blackbox Large Language Models without Logit Access
Constrained decoding, a technique for enforcing constraints on language model outputs, offers a way to control text generation without retraining or architectural modifications. Its application is, however, typically res…
Constituency ParsingLanguage ModelingLanguage ModellingText GenerationInterpretability of Blackbox Machine Learning Models through Dataview Extraction and Shadow Model creation
Deep learning models trained using massive amounts of data tend to capture one view of the data and its associated mapping. Different deep learning models built on the same training data may capture different views of th…
BIG-bench Machine LearningDeep LearningExplanation Generationnerblackbox: A High-level Library for Named Entity Recognition in Python
We present nerblackbox, a python library to facilitate the use of state-of-the-art transformer-based models for named entity recognition. It provides simple-to-use yet powerful methods to access data and models from a wi…
named-entity-recognitionNamed Entity RecognitionInterpreting Blackbox Models via Model Extraction
Interpretability has become incredibly important as machine learning is increasingly used to inform consequential decisions. We propose to construct global explanations of complex, blackbox models in the form of a decisi…
modelModel extraction