From Spatial Relations to Spatial Configurations
Spatial Reasoning from language is essential for natural language understanding. Supporting it requires a representation scheme that can capture spatial phenomena encountered in language as well as in images and videos. Existing spatial representations are not sufficient for describing spatial configurations used in complex tasks. This paper extends the capabilities of existing spatial representation languages and increases coverage of the semantic aspects that are needed to ground the spatial meaning of natural language text in the world. Our spatial relation language is able to represent a large, comprehensive set of spatial concepts crucial for reasoning and is designed to support the composition of static and dynamic spatial configurations. We integrate this language with the Abstract Meaning Representation(AMR) annotation schema and present a corpus annotated by this extended AMR. To exhibit the applicability of our representation scheme, we annotate text taken from diverse datasets and show how we extend the capabilities of existing spatial representation languages with the fine-grained decomposition of semantics and blend it seamlessly with AMRs of sentences and discourse representations as a whole.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstract Meaning RepresentationNatural Language UnderstandingSpatial ReasoningSimilar Papers 제목 키워드 기반
Hierarchical Spatial Sum-Product Networks for Action Recognition in Still Images
Recognizing actions from still images is popularly studied recently. In this paper, we model an action class as a flexible number of spatial configurations of body parts by proposing a new spatial SPN (Sum-Product Networ…
Action RecognitionAction Recognition In Still ImagesTemporal Action LocalizationSpatialNet: A Declarative Resource for Spatial Relations
This paper introduces SpatialNet, a novel resource which links linguistic expressions to actual spatial configurations. SpatialNet is based on FrameNet (Ruppenhofer et al., 2016) and VigNet (Coyne et al., 2011), two reso…
Automatic Speech Recognition (ASR)Image Annotation with ISO-Space: Distinguishing Content from Structure
Natural language descriptions of visual media present interesting problems for linguistic annotation of spatial information. This paper explores the use of ISO-Space, an annotation specification to capturing spatial info…
Content-Based Image RetrievalImage CaptioningImage RetrievalSemantic Role LabelingOpen-World Visual Reasoning by a Neuro-Symbolic Program of Zero-Shot Symbols
We consider the problem of finding spatial configurations of multiple objects in images, e.g., a mobile inspection robot is tasked to localize abandoned tools on the floor. We define the spatial configuration of objects …
Visual ReasoningCommonsense Spatial Reasoning for Visually Intelligent Agents
Service robots are expected to reliably make sense of complex, fast-changing environments. From a cognitive standpoint, they need the appropriate reasoning capabilities and background knowledge required to exhibit human-…
Spatial Reasoning