Sequential Manipulation Planning on Scene Graph

2022-07-10 · Ziyuan Jiao, Yida Niu, Zeyu Zhang, Song-Chun Zhu, Yixin Zhu, Hangxin Liu

We devise a 3D scene graph representation, contact graph+ (cg+), for efficient sequential task planning. Augmented with predicate-like attributes, this contact graph-based representation abstracts scene layouts with succinct geometric information and valid robot-scene interactions. Goal configurations, naturally specified on contact graphs, can be produced by a genetic algorithm with a stochastic optimization method. A task plan is then initialized by computing the Graph Editing Distance (GED) between the initial contact graphs and the goal configurations, which generates graph edit operations corresponding to possible robot actions. We finalize the task plan by imposing constraints to regulate the temporal feasibility of graph edit operations, ensuring valid task and motion correspondences. In a series of simulations and experiments, robots successfully complete complex sequential object rearrangement tasks that are difficult to specify using conventional planning language like Planning Domain Definition Language (PDDL), demonstrating the high feasibility and potential of robot sequential task planning on contact graph.

📄 PDF Abstract BibTeX arXiv:2207.04364

Code (1)

zyjiao4728/POG-Demo 공식 구현

Tasks

Object Rearrangement Stochastic OptimizationTask Planningvalid

Integrated Exploration and Sequential Manipulation on Scene Graph with LLM-based Situated Replanning

2026-02-04 · Heqing Yang, Ziyuan Jiao, Shu Wang, Yida Niu 외 arxiv

In partially known environments, robots must combine exploration to gather information with task planning for efficient execution. To address this challenge, we propose EPoG, an Exploration-based sequential manipulation …

Integration of Robot and Scene Kinematics for Sequential Mobile Manipulation Planning

2025-08-26 · Ziyuan Jiao, Yida Niu, Zeyu Zhang, Yangyang Wu 외 arxiv

We present a Sequential Mobile Manipulation Planning (SMMP) framework that can solve long-horizon multi-step mobile manipulation tasks with coordinated whole-body motion, even when interacting with articulated objects. B…

SAGE: Scene Graph-Aware Guidance and Execution for Long-Horizon Manipulation Tasks

2025-09-26 · Jialiang Li, Wenzheng Wu, Gaojing Zhang, Yifan Han 외 arxiv

Successfully solving long-horizon manipulation tasks remains a fundamental challenge. These tasks involve extended action sequences and complex object interactions, presenting a critical gap between high-level symbolic p…

Continuous Control Image Inpainting Image Editing

Learning Neural Implicit Functions as Object Representations for Robotic Manipulation

2021-09-29 · Jung-Su Ha, Danny Driess, Marc Toussaint

Robotic manipulation planning is the problem of finding a sequence of robot configurations that involves interactions with objects in the scene, e.g., grasp, placement, tool-use, etc. To achieve such interactions, tradit…

NeRF Open-Ended Question Answering Robot Manipulation

VeriGraph: Scene Graphs for Execution Verifiable Robot Planning

2024-11-15 · Daniel Ekpo, Mara Levy, Saksham Suri, Chuong Huynh 외

Recent advancements in vision-language models (VLMs) offer potential for robot task planning, but challenges remain due to VLMs' tendency to generate incorrect action sequences. To address these limitations, we propose V…

Robot Task Planning Task Planning