Building and Learning Structures in a Situated Blocks World Through Deep Language Understanding
We demonstrate a system for understanding natural language utterances for structure description and placement in a situated blocks world context. By relying on a rich, domain-specific adaptation of a generic ontology and a logical form structure produced by a semantic parser, we obviate the need for an intermediate, domain-specific representation and can produce a reasoner that grounds and reasons over concepts and constraints with real-valued data. This linguistic base enables more flexibility in interpreting natural language expressions invoking intrinsic concepts and features of structures and space. We demonstrate some of the capabilities of a system grounded in deep language understanding and present initial results in a structure learning task.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Back to the Blocks World: Learning New Actions through Situated Human-Robot Dialogue
A Situated Dialogue System for Learning Structural Concepts in Blocks World
We present a modular, end-to-end dialogue system for a situated agent to address a multimodal, natural language dialogue task in which the agent learns complex representations of block structure classes through assertion…
Transfer LearningMindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks
An ideal integration of autonomous agents in a human world implies that they are able to collaborate on human terms. In particular, theory of mind plays an important role in maintaining common ground during human collabo…
MinecraftTheory of Mind ModelingEquivariant Maps for Hierarchical Structures
While using invariant and equivariant maps, it is possible to apply deep learning to a range of primitive data structures, a formalism for dealing with hierarchy is lacking. This is a significant issue because many pract…
3D Semantic SegmentationSemantic SegmentationTranslationEquivariant Networks for Hierarchical Structures
While using invariant and equivariant maps, it is possible to apply deep learning to a range of primitive data structures, a formalism for dealing with hierarchy is lacking. This is a significant issue because many pract…
Semantic SegmentationTranslation