Reasoning with Scene Graphs for Robot Planning under Partial Observability
Robot planning in partially observable domains is difficult, because a robot needs to estimate the current state and plan actions at the same time. When the domain includes many objects, reasoning about the objects and their relationships makes robot planning even more difficult. In this paper, we develop an algorithm called scene analysis for robot planning (SARP) that enables robots to reason with visual contextual information toward achieving long-term goals under uncertainty. SARP constructs scene graphs, a factored representation of objects and their relations, using images captured from different positions, and reasons with them to enable context-aware robot planning under partial observability. Experiments have been conducted using multiple 3D environments in simulation, and a dataset collected by a real robot. In comparison to standard robot planning and scene analysis methods, in a target search domain, SARP improves both efficiency and accuracy in task completion. Supplementary material can be found at https://tinyurl.com/sarp22
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Queryable 3D Scene Representation: A Multi-Modal Framework for Semantic Reasoning and Robotic Task Planning
To enable robots to comprehend high-level human instructions and perform complex tasks, a key challenge lies in achieving comprehensive scene understanding: interpreting and interacting with the 3D environment in a meani…
Scene UnderstandingPoint CloudsLightweight Structured Multimodal Reasoning for Clinical Scene Understanding in Robotics
Healthcare robotics requires robust multimodal perception and reasoning to ensure safety in dynamic clinical environments. Current Vision-Language Models (VLMs) demonstrate strong general-purpose capabilities but remain …
Multimodal ReasoningScene UnderstandingLanguage-Grounded Hierarchical Planning and Execution with Multi-Robot 3D Scene Graphs
In this paper, we introduce a multi-robot system that integrates mapping, localization, and task and motion planning (TAMP) enabled by 3D scene graphs to execute complex instructions expressed in natural language. Our sy…
Language ModelingLanguage ModellingLarge Language ModelMotion Planning+1Belief Scene Graphs: Expanding Partial Scenes with Objects through Computation of Expectation
In this article, we propose the novel concept of Belief Scene Graphs, which are utility-driven extensions of partial 3D scene graphs, that enable efficient high-level task planning with partial information. We propose a …
Common Sense ReasoningTask PlanningRoBoSR: Structured Scene Representations for Embodied Robotic Reasoning
Despite rapid progress, embodied reasoning under real-world variability remains challenging. Existing approaches rely on demonstration-driven sequential biases, limiting flexibility in open-ended and long-horizon tasks t…
Zero-shot GeneralizationScene Understanding