Word2vec to behavior: morphology facilitates the grounding of language in machines
Enabling machines to respond appropriately to natural language commands could greatly expand the number of people to whom they could be of service. Recently, advances in neural network-trained word embeddings have empowered non-embodied text-processing algorithms, and suggest they could be of similar utility for embodied machines. Here we introduce a method that does so by training robots to act similarly to semantically-similar word2vec encoded commands. We show that this enables them to act appropriately, after training, to previously-unheard commands. Finally, we show that inducing such an alignment between motoric and linguistic similarities can be facilitated or hindered by the mechanical structure of the robot. This points to future, large scale methods that find and exploit relationships between action, language, and robot structure.
Code (1)
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
Evaluation of Morphological Embeddings for the Russian Language
A number of morphology-based word embedding models were introduced in recent years. However, their evaluation was mostly limited to English, which is known to be a morphologically simple language. In this paper, we explo…
ChunkingNERPOSPOS Tagging+1New word analogy corpus for exploring embeddings of Czech words
The word embedding methods have been proven to be very useful in many tasks of NLP (Natural Language Processing). Much has been investigated about word embeddings of English words and phrases, but only little attention h…
Word EmbeddingsEmbodied Multimodal Multitask Learning
Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for different multimodal tasks, such as semantic goal navigation and…
Deep Reinforcement LearningDisentanglementEmbodied Question AnsweringQuestion Answering+3LingoMotion: An Interpretable and Unambiguous Symbolic Representation for Human Motion
Existing representations for human motion, such as MotionGPT, often operate as black-box latent vectors with limited interpretability and build on joint positions which can cause ambiguity. Inspired by the hierarchical s…
Cross-Task Knowledge Transfer for Visually-Grounded Navigation
Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for two different tasks: learning to follow navigational instruction…
Deep Reinforcement LearningDisentanglementEmbodied Question AnsweringQuestion Answering+3