The red one!: On learning to refer to things based on their discriminative properties
As a first step towards agents learning to communicate about their visual environment, we propose a system that, given visual representations of a referent (cat) and a context (sofa), identifies their discriminative attributes, i.e., properties that distinguish them (has_tail). Moreover, despite the lack of direct supervision at the attribute level, the model learns to assign plausible attributes to objects (sofa-has_cushion). Finally, we present a preliminary experiment confirming the referential success of the predicted discriminative attributes.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeSimilar Papers 제목 키워드 기반
The red one!: On learning to refer to things based on discriminative properties
Searching Scenes by Abstracting Things
In this paper we propose to represent a scene as an abstraction of 'things'. We start from 'things' as generated by modern object proposals, and we investigate their immediately observable properties: position, size, asp…
AttributeDescriptiveObjectLess Descriptive yet Discriminative: Quantifying the Properties of Multimodal Referring Utterances via CLIP
In this work, we use a transformer-based pre-trained multimodal model, CLIP, to shed light on the mechanisms employed by human speakers when referring to visual entities. In particular, we use CLIP to quantify the degree…
DescriptiveKGTrust: Evaluating Trustworthiness of SIoT via Knowledge Enhanced Graph Neural Networks
Social Internet of Things (SIoT), a promising and emerging paradigm that injects the notion of social networking into smart objects (i.e., things), paving the way for the next generation of Internet of Things. However, d…
Graph Neural NetworkHow agents see things: On visual representations in an emergent language game
There is growing interest in the language developed by agents interacting in emergent-communication settings. Earlier studies have focused on the agents' symbol usage, rather than on their representation of visual input.…