Mastering emergent language: learning to guide in simulated navigation
To cooperate with humans effectively, virtual agents need to be able to understand and execute language instructions. A typical setup to achieve this is with a scripted teacher which guides a virtual agent using language instructions. However, such setup has clear limitations in scalability and, more importantly, it is not interactive. Here, we introduce an autonomous agent that uses discrete communication to interactively guide other agents to navigate and act on a simulated environment. The developed communication protocol is trainable, emergent and requires no additional supervision. The emergent language speeds up learning of new agents, it generalizes across incrementally more difficult tasks and, contrary to most other emergent languages, it is highly interpretable. We demonstrate how the emitted messages correlate with particular actions and observations, and how new agents become less dependent on this guidance as training progresses. By exploiting the correlations identified in our analysis, we manage to successfully address the agents in their own language.
Code (0)
등록된 구현이 없습니다.
Tasks
NavigateSimilar Papers 제목 키워드 기반
Mastering emergent language: learning to guide in simulated navigation
To cooperate with humans effectively, virtual agents need to be able to understand and execute language instructions. A typical setup to achieve this is with a scripted teacher which guides a virtual agent using language…
NavigateLearning to cooperate: Emergent communication in multi-agent navigation
Emergent communication in artificial agents has been studied to understand language evolution, as well as to develop artificial systems that learn to communicate with humans. We show that agents performing a cooperative …
Knowledge Distillation from Language-Oriented to Emergent Communication for Multi-Agent Remote Control
In this work, we compare emergent communication (EC) built upon multi-agent deep reinforcement learning (MADRL) and language-oriented semantic communication (LSC) empowered by a pre-trained large language model (LLM) usi…
Deep Reinforcement LearningKnowledge DistillationLanguage ModelingLanguage Modelling+2Situated Multimodal Control of a Mobile Robot: Navigation through a Virtual Environment
We present a new interface for controlling a navigation robot in novel environments using coordinated gesture and language. We use a TurtleBot3 robot with a LIDAR and a camera, an embodied simulation of what the robot ha…
Robot NavigationFrom Language to Action: Can LLM-Based Agents Be Used for Embodied Robot Cognition?
In order to flexibly act in an everyday environment, a robotic agent needs a variety of cognitive capabilities that enable it to reason about plans and perform execution recovery. Large language models (LLMs) have been s…
Instruction Following