High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization
We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views while preserving specific details of the input image. High-fidelity 3D GAN inversion is inherently challenging due to the geometry-texture trade-off in 3D inversion, where overfitting to a single view input image often damages the estimated geometry during the latent optimization. To solve this challenge, we propose a novel pipeline that builds on the pseudo-multi-view estimation with visibility analysis. We keep the original textures for the visible parts and utilize generative priors for the occluded parts. Extensive experiments show that our approach achieves advantageous reconstruction and novel view synthesis quality over state-of-the-art methods, even for images with out-of-distribution textures. The proposed pipeline also enables image attribute editing with the inverted latent code and 3D-aware texture modification. Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.
Code (1)
Tasks
AttributeGenerative Adversarial NetworkNovel View SynthesisVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
3D GAN Inversion with Facial Symmetry Prior
Recently, a surge of high-quality 3D-aware GANs have been proposed, which leverage the generative power of neural rendering. It is natural to associate 3D GANs with GAN inversion methods to project a real image into the …
3D geometryImage ReconstructionNeural RenderingWarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting
3D GAN inversion projects a single image into the latent space of a pre-trained 3D GAN to achieve single-shot novel view synthesis, which requires visible regions with high fidelity and occluded regions with realism and …
Novel View SynthesisImage InpaintingTextual Inversion and Self-supervised Refinement for Radiology Report Generation
Existing mainstream approaches follow the encoder-decoder paradigm for generating radiology reports. They focus on improving the network structure of encoders and decoders, which leads to two shortcomings: overlooking th…
DecoderHigh-Fidelity GAN Inversion for Image Attribute Editing
We present a novel high-fidelity generative adversarial network (GAN) inversion framework that enables attribute editing with image-specific details well-preserved (e.g., background, appearance, and illumination). We fir…
AttributeGenerative Adversarial NetworkImage ManipulationVocal Bursts Intensity PredictionControllable 3D Object Generation with Single Image Prompt
Recently, the impressive generative capabilities of diffusion models have been demonstrated, producing images with remarkable fidelity. Particularly, existing methods for the 3D object generation tasks, which is one of t…