Planning with Spatial-Temporal Abstraction from Point Clouds for Deformable Object Manipulation
Effective planning of long-horizon deformable object manipulation requires suitable abstractions at both the spatial and temporal levels. Previous methods typically either focus on short-horizon tasks or make strong assumptions that full-state information is available, which prevents their use on deformable objects. In this paper, we propose PlAnning with Spatial-Temporal Abstraction (PASTA), which incorporates both spatial abstraction (reasoning about objects and their relations to each other) and temporal abstraction (reasoning over skills instead of low-level actions). Our framework maps high-dimension 3D observations such as point clouds into a set of latent vectors and plans over skill sequences on top of the latent set representation. We show that our method can effectively perform challenging sequential deformable object manipulation tasks in the real world, which require combining multiple tool-use skills such as cutting with a knife, pushing with a pusher, and spreading the dough with a roller.
Code (0)
등록된 구현이 없습니다.
Tasks
Deformable Object ManipulationSimilar Papers 제목 키워드 기반
FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Clouds
Scene flow depicts the dynamics of a 3D scene, which is critical for various applications such as autonomous driving, robot navigation, AR/VR, etc. Conventionally, scene flow is estimated from dense/regular RGB video fra…
Autonomous DrivingRobot NavigationScene Flow EstimationSequential 3D Human Pose and Shape Estimation From Point Clouds
This work addresses the problem of 3D human pose and shape estimation from a sequence of point clouds. Existing sequential 3D human shape estimation methods mainly focus on the template model fitting from a sequence of d…
3D human pose and shape estimation3D Human Shape EstimationSelf-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks
Exploiting past 3D LiDAR scans to predict future point clouds is a promising method for autonomous mobile systems to realize foresighted state estimation, collision avoidance, and planning. In this paper, we address the …
Collision AvoidanceDecoderState EstimationSpSequenceNet: Semantic Segmentation Network on 4D Point Clouds
Point clouds are useful in many applications like autonomous driving and robotics as they provide natural 3D information of the surrounding environments. While there are extensive research on 3D point clouds, scene under…
Autonomous DrivingScene UnderstandingSemantic SegmentationUnsupervised Learning of Global Registration of Temporal Sequence of Point Clouds
Global registration of point clouds aims to find an optimal alignment of a sequence of 2D or 3D point sets. In this paper, we present a novel method that takes advantage of current deep learning techniques for unsupervis…