Registered and Segmented Deformable Object Reconstruction from a Single View Point Cloud
In deformable object manipulation, we often want to interact with specific segments of an object that are only defined in non-deformed models of the object. We thus require a system that can recognize and locate these segments in sensor data of deformed real world objects. This is normally done using deformable object registration, which is problem specific and complex to tune. Recent methods utilize neural occupancy functions to improve deformable object registration by registering to an object reconstruction. Going one step further, we propose a system that in addition to reconstruction learns segmentation of the reconstructed object. As the resulting output already contains the information about the segments, we can skip the registration process. Tested on a variety of deformable objects in simulation and the real world, we demonstrate that our method learns to robustly find these segments. We also introduce a simple sampling algorithm to generate better training data for occupancy learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Deformable Object ManipulationObjectObject ReconstructionSimilar Papers 제목 키워드 기반
3DFS: Deformable Dense Depth Fusion and Segmentation for Object Reconstruction from a Handheld Camera
We propose an approach for 3D reconstruction and segmentation of a single object placed on a flat surface from an input video. Our approach is to perform dense depth map estimation for multiple views using a proposed obj…
3D ReconstructionDepth EstimationObjectObject Reconstruction+2Multi-modal deformable image registration using untrained neural networks
Image registration techniques usually assume that the images to be registered are of a certain type (e.g. single- vs. multi-modal, 2D vs. 3D, rigid vs. deformable) and there lacks a general method that can work for data …
Image RegistrationHMDO: Markerless Multi-view Hand Manipulation Capture with Deformable Objects
We construct the first markerless deformable interaction dataset recording interactive motions of the hands and deformable objects, called HMDO (Hand Manipulation with Deformable Objects). With our built multi-view captu…
ObjectObject ReconstructionPose EstimationDOFS: A Real-world 3D Deformable Object Dataset with Full Spatial Information for Dynamics Model Learning
This work proposes DOFS, a pilot dataset of 3D deformable objects (DOs) (e.g., elasto-plastic objects) with full spatial information (i.e., top, side, and bottom information) using a novel and low-cost data collection pl…
Self-supervised Single-view 3D Reconstruction via Semantic Consistency
We learn a self-supervised, single-view 3D reconstruction model that predicts the 3D mesh shape, texture and camera pose of a target object with a collection of 2D images and silhouettes. The proposed method does not nec…
3D ReconstructionObjectSingle-View 3D Reconstruction