AugGS: Self-augmented Gaussians with Structural Masks for Sparse-view 3D Reconstruction
Sparse-view 3D reconstruction is a major challenge in computer vision, aiming to create complete three-dimensional models from limited viewing angles. Key obstacles include: 1) a small number of input images with inconsistent information; 2) dependence on input image quality; and 3) large model parameter sizes. To tackle these issues, we propose a self-augmented two-stage Gaussian splatting framework enhanced with structural masks for sparse-view 3D reconstruction. Initially, our method generates a basic 3D Gaussian representation from sparse inputs and renders multi-view images. We then fine-tune a pre-trained 2D diffusion model to enhance these images, using them as augmented data to further optimize the 3D Gaussians. Additionally, a structural masking strategy during training enhances the model's robustness to sparse inputs and noise. Experiments on benchmarks like MipNeRF360, OmniObject3D, and OpenIllumination demonstrate that our approach achieves state-of-the-art performance in perceptual quality and multi-view consistency with sparse inputs.
Code (0)
등록된 구현이 없습니다.
Tasks
3D geometry3D ReconstructionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning
In math reasoning with large language models (LLMs), fine-tuning data augmentation by query evolution and diverse reasoning paths is empirically verified effective, profoundly narrowing the gap between open-sourced LLMs …
Arithmetic ReasoningData AugmentationGSM8KMath+2Contrastive Gaussian Clustering: Weakly Supervised 3D Scene Segmentation
We introduce Contrastive Gaussian Clustering, a novel approach capable of provide segmentation masks from any viewpoint and of enabling 3D segmentation of the scene. Recent works in novel-view synthesis have shown how to…
ClusteringContrastive LearningNovel View SynthesisScene Segmentation+1DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes
We present DeSiRe-GS, a self-supervised gaussian splatting representation, enabling effective static-dynamic decomposition and high-fidelity surface reconstruction in complex driving scenarios. Our approach employs a two…
Autonomous DrivingSurface ReconstructionMaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks
While 3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in novel view synthesis and real-time rendering, the high memory consumption due to the use of millions of Gaussians limits its practicality. To …
3DGSNovel View SynthesisSA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection
We propose Self-Augmented Residual 3D Gaussian Splatting (SA-ResGS), a novel framework to stabilize uncertainty quantification and enhancing uncertainty-aware supervision in next-best-view (NBV) selection for active scen…
Point Clouds