Depth-Guided Robust and Fast Point Cloud Fusion NeRF for Sparse Input Views
Novel-view synthesis with sparse input views is important for real-world applications like AR/VR and autonomous driving. Recent methods have integrated depth information into NeRFs for sparse input synthesis, leveraging depth prior for geometric and spatial understanding. However, most existing works tend to overlook inaccuracies within depth maps and have low time efficiency. To address these issues, we propose a depth-guided robust and fast point cloud fusion NeRF for sparse inputs. We perceive radiance fields as an explicit voxel grid of features. A point cloud is constructed for each input view, characterized within the voxel grid using matrices and vectors. We accumulate the point cloud of each input view to construct the fused point cloud of the entire scene. Each voxel determines its density and appearance by referring to the point cloud of the entire scene. Through point cloud fusion and voxel grid fine-tuning, inaccuracies in depth values are refined or substituted by those from other views. Moreover, our method can achieve faster reconstruction and greater compactness through effective vector-matrix decomposition. Experimental results underline the superior performance and time efficiency of our approach compared to state-of-the-art baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingNeRFNovel View SynthesisSimilar Papers 제목 키워드 기반
Volumetric Propagation Network: Stereo-LiDAR Fusion for Long-Range Depth Estimation
Stereo-LiDAR fusion is a promising task in that we can utilize two different types of 3D perceptions for practical usage -- dense 3D information (stereo cameras) and highly-accurate sparse point clouds (LiDAR). However, …
Depth CompletionDepth EstimationSensor FusionStereo-LiDAR Fusion+2TriFusion-AE: Language-Guided Depth and LiDAR Fusion for Robust Point Cloud Processing
LiDAR-based perception is central to autonomous driving and robotics, yet raw point clouds remain highly vulnerable to noise, occlusion, and adversarial corruptions. Autoencoders offer a natural framework for denoising a…
Representation LearningAutonomous DrivingPoint CloudsSFGFusion: Surface Fitting Guided 3D Object Detection with 4D Radar and Camera Fusion
3D object detection is essential for autonomous driving. As an emerging sensor, 4D imaging radar offers advantages as low cost, long-range detection, and accurate velocity measurement, making it highly suitable for objec…
3D Object DetectionAutonomous DrivingPoint CloudsDepth-Semantic Alignment and Affinity-Guided Fusion for Structured Radar Point Cloud Generation
Point clouds are an important carrier of three-dimensional spatial information, and their quality directly affects the performance of downstream perception tasks such as object detection and tracking. However, millimeter…
Point Cloud GenerationObject DetectionPoint CloudsPoint Cloud Completion with Pretrained Text-to-Image Diffusion Models
Point cloud data collected in real-world applications are often incomplete. This is because they are observed from partial viewpoints, which capture only a specific perspective or angle, or due to occlusion and low resol…