Generating Quantified Referring Expressions through Attention-Driven Incremental Perception
We model the production of quantified referring expressions (QREs) that identity collections of visual items. A previous approach, called Perceptual Cost Pruning, modeled human QRE production using a preference-based referring expression generation algorithm, first removing facts from the input knowledge base based on a model of perceptual cost. In this paper, we present an alternative model that incrementally constructs a symbolic knowledge base through simulating human visual attention/perception from raw images. We demonstrate that this model produces the same output as Perceptual Cost Pruning. We argue that this is a more extensible approach and a step toward developing a wider range of process-level models of human visual description.
Code (0)
등록된 구현이 없습니다.
Tasks
Referring ExpressionReferring expression generationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Generating Quantified Descriptions of Abstract Visual Scenes
Quantified expressions have always taken up a central position in formal theories of meaning and language use. Yet quantified expressions have so far attracted far less attention from the Natural Language Generation comm…
PositionText GenerationGenerating Quantified Referring Expressions with Perceptual Cost Pruning
We model the production of quantified referring expressions (QREs) that identify collections of visual items. To address this task, we propose a method of perceptual cost pruning, which consists of two steps: (1) determi…
Generation of Quantified Referring Expressions: Evidence from Experimental Data
Talking about what is not there: Generating indefinite referring expressions in Minecraft
When generating technical instructions, it is often necessary to describe an object that does not exist yet. For example, an NLG system which explains how to build a house needs to generate sentences like {``}build *a wa…
MinecraftObjectGenerating a Novel Dataset of Multimodal Referring Expressions
Referring expressions and definite descriptions of objects in space exploit information both about object characteristics and locations. To resolve potential ambiguity, referencing strategies in language can rely on incr…