LIST: Learning Implicitly from Spatial Transformers for Single-View 3D Reconstruction
Accurate reconstruction of both the geometric and topological details of a 3D object from a single 2D image embodies a fundamental challenge in computer vision. Existing explicit/implicit solutions to this problem struggle to recover self-occluded geometry and/or faithfully reconstruct topological shape structures. To resolve this dilemma, we introduce LIST, a novel neural architecture that leverages local and global image features to accurately reconstruct the geometric and topological structure of a 3D object from a single image. We utilize global 2D features to predict a coarse shape of the target object and then use it as a base for higher-resolution reconstruction. By leveraging both local 2D features from the image and 3D features from the coarse prediction, we can predict the signed distance between an arbitrary point and the target surface via an implicit predictor with great accuracy. Furthermore, our model does not require camera estimation or pixel alignment. It provides an uninfluenced reconstruction from the input-view direction. Through qualitative and quantitative analysis, we show the superiority of our model in reconstructing 3D objects from both synthetic and real-world images against the state of the art.
Code (1)
Tasks
3D ReconstructionObjectSingle-View 3D ReconstructionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Geometry-Free View Synthesis: Transformers and no 3D Priors
Is a geometric model required to synthesize novel views from a single image? Being bound to local convolutions, CNNs need explicit 3D biases to model geometric transformations. In contrast, we demonstrate that a transfor…
Novel View SynthesisMulti-view Action Recognition via Directed Gromov-Wasserstein Discrepancy
Action recognition has become one of the popular research topics in computer vision. There are various methods based on Convolutional Networks and self-attention mechanisms as Transformers to solve both spatial and tempo…
Action RecognitionFrom Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Models
While multimodal large language models (MLLMs) have made substantial progress in single-image spatial reasoning, multi-image spatial reasoning, which requires integration of information from multiple viewpoints, remains …
Spatial ReasoningEpipolar Attention Field Transformers for Bird's Eye View Semantic Segmentation
Spatial understanding of the semantics of the surroundings is a key capability needed by autonomous cars to enable safe driving decisions. Recently, purely vision-based solutions have gained increasing research interest.…
Bird's-Eye View Semantic SegmentationSemantic SegmentationGitNet: Geometric Prior-based Transformation for Birds-Eye-View Segmentation
Birds-eye-view (BEV) semantic segmentation is critical for autonomous driving for its powerful spatial representation ability. It is challenging to estimate the BEV semantic maps from monocular images due to the spatial …
Autonomous DrivingBEV SegmentationImage SegmentationSegmentation+1