Estimating Commonsense Scene Composition on Belief Scene Graphs
This work establishes the concept of commonsense scene composition, with a focus on extending Belief Scene Graphs by estimating the spatial distribution of unseen objects. Specifically, the commonsense scene composition capability refers to the understanding of the spatial relationships among related objects in the scene, which in this article is modeled as a joint probability distribution for all possible locations of the semantic object class. The proposed framework includes two variants of a Correlation Information (CECI) model for learning probability distributions: (i) a baseline approach based on a Graph Convolutional Network, and (ii) a neuro-symbolic extension that integrates a spatial ontology based on Large Language Models (LLMs). Furthermore, this article provides a detailed description of the dataset generation process for such tasks. Finally, the framework has been validated through multiple runs on simulated data, as well as in a real-world indoor environment, demonstrating its ability to spatially interpret scenes across different room types.
Code (0)
등록된 구현이 없습니다.
Tasks
Dataset GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning Visual Commonsense for Robust Scene Graph Generation
Scene graph generation models understand the scene through object and predicate recognition, but are prone to mistakes due to the challenges of perception in the wild. Perception errors often lead to nonsensical composit…
Graph GenerationScene Graph GenerationScene UnderstandingSpatial Commonsense Graph for Object Localisation in Partial Scenes
We solve object localisation in partial scenes, a new problem of estimating the unknown position of an object (e.g. where is the bag?) given a partial 3D scan of a scene. The proposed solution is based on a novel scene g…
Graph Neural NetworkObjectPositionGenerative Visual Commonsense Answering and Explaining with Generative Scene Graph Constructing
Visual Commonsense Reasoning, which is regarded as one challenging task to pursue advanced visual scene comprehension, has been used to diagnose the reasoning ability of AI systems. However, reliable reasoning requires a…
Visual Commonsense ReasoningCRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense
Alternatively inferring on the visual facts and commonsense is fundamental for an advanced VQA system. This ability requires models to go beyond the literal understanding of commonsense. The system should not just treat …
Question AnsweringVisual Question Answering (VQA)SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning
Answering complex questions about images is an ambitious goal for machine intelligence, which requires a joint understanding of images, text, and commonsense knowledge, as well as a strong reasoning ability. Recently, mu…
Visual Commonsense Reasoning