Pointmap-Conditioned Diffusion for Consistent Novel View Synthesis
In this paper, we present PointmapDiffusion, a novel framework for single-image novel view synthesis (NVS) that utilizes pre-trained 2D diffusion models. Our method is the first to leverage pointmaps (i.e. rasterized 3D scene coordinates) as a conditioning signal, capturing geometric prior from the reference images to guide the diffusion process. By embedding reference attention blocks and a ControlNet for pointmap features, our model balances between generative capability and geometric consistency, enabling accurate view synthesis across varying viewpoints. Extensive experiments on diverse real-world datasets demonstrate that PointmapDiffusion achieves high-quality, multi-view consistent results with significantly fewer trainable parameters compared to other baselines for single-image NVS tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Novel View SynthesisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Outdoor Monocular SLAM with Global Scale-Consistent 3D Gaussian Pointmaps
3D Gaussian Splatting (3DGS) has become a popular solution in SLAM due to its high-fidelity and real-time novel view synthesis performance. However, some previous 3DGS SLAM methods employ a differentiable rendering pipel…
3DGSNovel View SynthesisRGB-Only Gaussian Splatting SLAM for Unbounded Outdoor Scenes
3D Gaussian Splatting (3DGS) has become a popular solution in SLAM, as it can produce high-fidelity novel views. However, previous GS-based methods primarily target indoor scenes and rely on RGB-D sensors or pre-trained …
3DGSCamera Pose EstimationDepth EstimationNovel View Synthesis+2unPIC: A Geometric Multiview Prior for Image to 3D Synthesis
We introduce a hierarchical probabilistic approach to go from a 2D image to multiview 3D: a diffusion "prior" predicts the unseen 3D geometry, which then conditions a diffusion "decoder" to generate novel views of the su…
3D geometryDecoderImage to 3DNovel View SynthesisHumanGif: Single-View Human Diffusion with Generative Prior
While previous single-view-based 3D human reconstruction methods made significant progress in novel view synthesis, it remains a challenge to synthesize both view-consistent and pose-consistent results for animatable hum…
3D Human ReconstructionNeRFNovel View SynthesisViVid-1-to-3: Novel View Synthesis with Video Diffusion Models
Generating novel views of an object from a single image is a challenging task. It requires an understanding of the underlying 3D structure of the object from an image and rendering high-quality, spatially consistent new …
Novel View SynthesisObject