View-volume Network for Semantic Scene Completion from a Single Depth Image
We introduce a View-Volume convolutional neural network (VVNet) for inferring the occupancy and semantic labels of a volumetric 3D scene from a single depth image. The VVNet concatenates a 2D view CNN and a 3D volume CNN with a differentiable projection layer. Given a single RGBD image, our method extracts the detailed geometric features from the input depth image with a 2D view CNN and then projects the features into a 3D volume according to the input depth map via a projection layer. After that, we learn the 3D context information of the scene with a 3D volume CNN for computing the result volumetric occupancy and semantic labels. With combined 2D and 3D representations, the VVNet efficiently reduces the computational cost, enables feature extraction from multi-channel high resolution inputs, and thus significantly improves the result accuracy. We validate our method and demonstrate its efficiency and effectiveness on both synthetic SUNCG and real NYU dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Semantic Scene CompletionSimilar Papers 제목 키워드 기반
Point Cloud Scene Completion with Joint Color and Semantic Estimation from Single RGB-D Image
We present a deep reinforcement learning method of progressive view inpainting for colored semantic point cloud scene completion under volume guidance, achieving high-quality scene reconstruction from only a single RGB-D…
Deep Reinforcement LearningImage InpaintingSegmentationSemantic SegmentationSemantic Scene Completion from a Single Depth Image
This paper focuses on semantic scene completion, a task for producing a complete 3D voxel representation of volumetric occupancy and semantic labels for a scene from a single-view depth map observation. Previous work has…
3D Semantic Scene CompletionEdgeNet: Semantic Scene Completion from a Single RGB-D Image
Semantic scene completion is the task of predicting a complete 3D representation of volumetric occupancy with corresponding semantic labels for a scene from a single point of view. Previous works on Semantic Scene Comple…
3D Semantic Scene CompletionEdge DetectionSemantic Scene Completion Combining Colour and Depth: preliminary experiments
Semantic scene completion is the task of producing a complete 3D voxel representation of volumetric occupancy with semantic labels for a scene from a single-view observation. We built upon the recent work of Song et al. …
3D Semantic Scene CompletionOne at a Time: Progressive Multi-step Volumetric Probability Learning for Reliable 3D Scene Perception
Numerous studies have investigated the pivotal role of reliable 3D volume representation in scene perception tasks, such as multi-view stereo (MVS) and semantic scene completion (SSC). They typically construct 3D probabi…
Depth EstimationRepresentation Learning