Decoupling Pragmatics: Discriminative Decoding for Referring Expression Generation
The shift to neural models in Referring Expression Generation (REG) has enabled more natural set-ups, but at the cost of interpretability. We argue that integrating pragmatic reasoning into the inference of context-agnostic generation models could reconcile traits of traditional and neural REG, as this offers a separation between context-independent, literal information and pragmatic adaptation to context. With this in mind, we apply existing decoding strategies from discriminative image captioning to REG and evaluate them in terms of pragmatic informativity, likelihood to ground-truth annotations and linguistic diversity. Our results show general effectiveness, but a relatively small gain in informativity, raising important questions for REG in general.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityImage CaptioningReferring ExpressionReferring expression generationSimilar Papers 제목 키워드 기반
Referring Expression Generation in Visually Grounded Dialogue with Discourse-aware Comprehension Guiding
We propose an approach to referring expression generation (REG) in visually grounded dialogue that is meant to produce referring expressions (REs) that are both discriminative and discourse-appropriate. Our method consti…
Image RetrievalReferring ExpressionReferring expression generationRetrievalReasoning About Pragmatics with Neural Listeners and Speakers
We present a model for pragmatically describing scenes, in which contrastive behavior results from a combination of inference-driven pragmatics and learned semantics. Like previous learned approaches to language generati…
Referring ExpressionText GenerationA Joint Speaker-Listener-Reinforcer Model for Referring Expressions
Referring expressions are natural language constructions used to identify particular objects within a scene. In this paper, we propose a unified framework for the tasks of referring expression comprehension and generatio…
Referring ExpressionReferring Expression ComprehensionAn Incremental Iterated Response Model of Pragmatics
Recent Iterated Response (IR) models of pragmatics conceptualize language use as a recursive process in which agents reason about each other to increase communicative efficiency. These models are generally defined over c…
modelReferring ExpressionReferring expression generationDecoding Strategies for Neural Referring Expression Generation
RNN-based sequence generation is now widely used in NLP and NLG (natural language generation). Most work focusses on how to train RNNs, even though also decoding is not necessarily straightforward: previous work on neura…
Image CaptioningMachine TranslationReferring ExpressionReferring expression generation+1