Structured Prediction of Unobserved Voxels From a Single Depth Image
Building a complete 3D model of a scene, given only a single depth image, is underconstrained. To gain a full volumetric model, one needs either multiple views, or a single view together with a library of unambiguous 3D models that will fit the shape of each individual object in the scene. We hypothesize that objects of dissimilar semantic classes often share similar 3D shape components, enabling a limited dataset to model the shape of a wide range of objects, and hence estimate their hidden geometry. Exploring this hypothesis, we propose an algorithm that can complete the unobserved geometry of tabletop-sized objects, based on a supervised model trained on already available volumetric elements. Our model maps from a local observation in a single depth image to an estimate of the surface shape in the surrounding neighborhood. We validate our approach both qualitatively and quantitatively on a range of indoor object collections and challenging real scenes.
Code (0)
등록된 구현이 없습니다.
Tasks
Structured PredictionSimilar Papers 제목 키워드 기반
Semantic Scene Completion with Cleaner Self
Semantic Scene Completion (SSC) transforms an image of single-view depth and/or RGB 2D pixels into 3D voxels, each of whose semantic labels are predicted. SSC is a well-known ill-posed problem as the prediction model has…
Pixels, voxels, and views: A study of shape representations for single view 3D object shape prediction
The goal of this paper is to compare surface-based and volumetric 3D object shape representations, as well as viewer-centered and object-centered reference frames for single-view 3D shape prediction. We propose a new alg…
ObjectHierarchical Surface Prediction for 3D Object Reconstruction
Recently, Convolutional Neural Networks have shown promising results for 3D geometry prediction. They can make predictions from very little input data such as a single color image. A major limitation of such approaches i…
3D geometry3D Geometry Prediction3D Object ReconstructionObject+2GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure
3D object detection from LiDAR point clouds is a core problem in autonomous driving. Recent advances in self-supervised learning (SSL) enable scalable pretraining and transfers well to per-point tasks such as semantic an…
Self-Supervised LearningRepresentation LearningPanoptic Segmentation3D Object DetectionEfficient Semantic Scene Completion Network with Spatial Group Convolution
We introduce Spatial Group Convolution (SGC) for accelerating the computation of 3D dense prediction tasks. SGC is orthogonal to group convolution, which works on spatial dimensions rather than feature channel dimension.…
3D Semantic Scene Completionvalid