Grasp, See, and Place: Efficient Unknown Object Rearrangement with Policy Structure Prior
We focus on the task of unknown object rearrangement, where a robot is supposed to re-configure the objects into a desired goal configuration specified by an RGB-D image. Recent works explore unknown object rearrangement systems by incorporating learning-based perception modules. However, they are sensitive to perception error, and pay less attention to task-level performance. In this paper, we aim to develop an effective system for unknown object rearrangement amidst perception noise. We theoretically reveal that the noisy perception impacts grasp and place in a decoupled way, and show such a decoupled structure is valuable to improve task optimality. We propose GSP, a dual-loop system with the decoupled structure as prior. For the inner loop, we learn a see policy for self-confident in-hand object matching. For the outer loop, we learn a grasp policy aware of object matching and grasp capability guided by task-level rewards. We leverage the foundation model CLIP for object matching, policy learning and self-termination. A series of experiments indicate that GSP can conduct unknown object rearrangement with higher completion rates and fewer steps.
Code (1)
Tasks
ObjectObject RearrangementMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Object Rearrangement Using Learned Implicit Collision Functions
Robotic object rearrangement combines the skills of picking and placing objects. When object models are unavailable, typical collision-checking models may be unable to predict collisions in partial point clouds with occl…
ObjectObject RearrangementHierarchical Policy for Non-prehensile Multi-object Rearrangement with Deep Reinforcement Learning and Monte Carlo Tree Search
Non-prehensile multi-object rearrangement is a robotic task of planning feasible paths and transferring multiple objects to their predefined target poses without grasping. It needs to consider how each object reaches the…
Deep Reinforcement LearningObjectObject RearrangementPush-Placement: A Hybrid Approach Integrating Prehensile and Non-Prehensile Manipulation for Object Rearrangement
Efficient tabletop rearrangement remains challenging due to collisions and the need for temporary buffering when target poses are obstructed. Prehensile pick-and-place provides precise control but often requires extra mo…
Object RearrangementTowards Precise Robotic Grasping by Probabilistic Post-grasp Displacement Estimation
Precise robotic grasping is important for many industrial applications, such as assembly and palletizing, where the location of the object needs to be controlled and known. However, achieving precise grasps is challengin…
ObjectRobotic GraspingOpen-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning
Detecting both known and unknown objects is a fundamental skill for robot manipulation in unstructured environments. Open-set object detection (OSOD) is a promising direction to handle the problem consisting of two subta…
Contrastive LearningObjectobject-detectionObject Detection+2