How agents see things: On visual representations in an emergent language game
There is growing interest in the language developed by agents interacting in emergent-communication settings. Earlier studies have focused on the agents' symbol usage, rather than on their representation of visual input. In this paper, we consider the referential games of Lazaridou et al. (2017) and investigate the representations the agents develop during their evolving interaction. We find that the agents establish successful communication by inducing visual representations that almost perfectly align with each other, but, surprisingly, do not capture the conceptual properties of the objects depicted in the input images. We conclude that, if we are interested in developing language-like communication systems, we must pay more attention to the visual semantics agents associate to the symbols they use.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Emergent Communication between Heterogeneous Visual Agents through Decentralized Learning
Symbols are shared, but perception is private. We study emergent communication between heterogeneous visual agents through decentralized learning, asking what visual information can become shareable when agents have diff…
Text RetrievalEC^2: Emergent Communication for Embodied Control
Embodied control requires agents to leverage multi-modal pre-training to quickly learn how to act in new environments, where video demonstrations contain visual and motion details needed for low-level perception and cont…
Contrastive LearningLanguage ModellingEC2: Emergent Communication for Embodied Control
Embodied control requires agents to leverage multi-modal pre-training to quickly learn how to act in new environments, where video demonstrations contain visual and motion details needed for low-level perception and …
Contrastive LearningLanguage ModellingEmergent Communication Protocol Learning for Task Offloading in Industrial Internet of Things
In this paper, we leverage a multi-agent reinforcement learning (MARL) framework to jointly learn a computation offloading decision and multichannel access policy with corresponding signaling. Specifically, the base stat…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningAvoiding hashing and encouraging visual semantics in referential emergent language games
There has been an increasing interest in the area of emergent communication between agents which learn to play referential signalling games with realistic images. In this work, we consider the signalling game setting of …