Associative3D: Volumetric Reconstruction from Sparse Views
This paper studies the problem of 3D volumetric reconstruction from two views of a scene with an unknown camera. While seemingly easy for humans, this problem poses many challenges for computers since it requires simultaneously reconstructing objects in the two views while also figuring out their relationship. We propose a new approach that estimates reconstructions, distributions over the camera/object and camera/camera transformations, as well as an inter-view object affinity matrix. This information is then jointly reasoned over to produce the most likely explanation of the scene. We train and test our approach on a dataset of indoor scenes, and rigorously evaluate the merits of our joint reasoning approach. Our experiments show that it is able to recover reasonable scenes from sparse views, while the problem is still challenging. Project site: https://jasonqsy.github.io/Associative3D
Code (1)
Tasks
3D Volumetric ReconstructionSimilar Papers 제목 키워드 기반
Deep Volumetric Video From Very Sparse Multi-View Performance Capture
We present a deep learning-based volumetric capture approach for performance capture using a passive and highly sparse multi-view capture system. We focus on a template-free, per-frame 3D surface reconstruction from as f…
Surface ReconstructionKeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints
Image-based volumetric humans using pixel-aligned features promise generalization to unseen poses and identities. Prior work leverages global spatial encodings and multi-view geometric consistency to reduce spatial ambig…
3D Face Reconstruction3D Human ReconstructionGeneralizable Novel View SynthesisNovel View SynthesisONIX: an X-ray deep-learning tool for 3D reconstructions from sparse views
Three-dimensional (3D) X-ray imaging techniques like tomography and confocal microscopy are crucial for academic and industrial applications. These approaches access 3D information by scanning the sample with respect to …
3D Object Reconstruction3D ReconstructionObject-Centric Multi-View Aggregation
We present an approach for aggregating a sparse set of views of an object in order to compute a semi-implicit 3D representation in the form of a volumetric feature grid. Key to our approach is an object-centric canonical…
Camera Pose EstimationNovel View SynthesisObjectPose EstimationHiReFF: High-Resolution Feedforward Human Reconstruction from Uncalibrated Sparse-View Video
Uncalibrated volumetric video streaming for human reconstruction is essential for holographic communication and AR/VR, yet remains challenging due to the need for temporal consistency and computational efficiency from sp…
Computational EfficiencyVideo Reconstruction