NeuralBlox: Real-Time Neural Representation Fusion for Robust Volumetric Mapping
We present a novel 3D mapping method leveraging the recent progress in neural implicit representation for 3D reconstruction. Most existing state-of-the-art neural implicit representation methods are limited to object-level reconstructions and can not incrementally perform updates given new data. In this work, we propose a fusion strategy and training pipeline to incrementally build and update neural implicit representations that enable the reconstruction of large scenes from sequential partial observations. By representing an arbitrarily sized scene as a grid of latent codes and performing updates directly in latent space, we show that incrementally built occupancy maps can be obtained in real-time even on a CPU. Compared to traditional approaches such as Truncated Signed Distance Fields (TSDFs), our map representation is significantly more robust in yielding a better scene completeness given noisy inputs. We demonstrate the performance of our approach in thorough experimental validation on real-world datasets with varying degrees of added pose noise.
Code (1)
Tasks
3D ReconstructionCPUSimilar Papers 제목 키워드 기반
PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation
We present PrimDiffusion, the first diffusion-based framework for 3D human generation. Devising diffusion models for 3D human generation is difficult due to the intensive computational cost of 3D representations and the …
3D InpaintingDecoderDenoisingCloak of Invisibility: Real-Time Privacy-Preserving Volumetric Video Streaming
Volumetric video streaming turns privacy into a 3D, multi-view problem. Unlike ordinary video, where sensitive content can often be redacted frame by frame, RGB-D volumetric pipelines capture people, rooms, and personal …
Object DetectionPoint CloudsGSFusion: Online RGB-D Mapping Where Gaussian Splatting Meets TSDF Fusion
Traditional volumetric fusion algorithms preserve the spatial structure of 3D scenes, which is beneficial for many tasks in computer vision and robotics. However, they often lack realism in terms of visualization. Emergi…
Computational EfficiencyNormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning
Multiview shape-from-shading (SfS) has achieved high-detail geometry, but its computation is expensive for solving a multiview registration and an ill-posed inverse rendering problem. Therefore, it has been mainly us…
Inverse RenderingDiff-Ensembler: Learning to Ensemble 2D Diffusion Models for Volume-to-Volume Medical Image Translation
Despite success in volume-to-volume translations in medical images, most existing models struggle to effectively capture the inherent volumetric distribution using 3D representations. The current state-of-the-art approac…
Image Super-ResolutionSuper-ResolutionTumor Segmentation