Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Network
3D-aware Generative Adversarial Networks (3D-GANs) currently exhibit artifacts in their 3D geometrical modeling, such as mesh imperfections and holes. These shortcomings are primarily attributed to the limited availability of annotated 3D data, leading to a constrained "valid latent area" for satisfactory modeling. To address this, we present a Self-Supervised Learning (SSL) technique tailored as an auxiliary loss for any 3D-GAN, designed to improve its 3D geometrical modeling capabilities. Our approach pioneers an inversion technique for 3D-GANs, integrating an encoder that performs adaptive spatially-varying range operations. Utilizing this inversion, we introduce the Cyclic Generative Constraint (CGC), aiming to densify the valid latent space. The CGC operates via augmented local latent vectors that maintain the same geometric form, and it imposes constraints on the cycle path outputs, specifically the generator-encoder-generator sequence. This SSL methodology seamlessly integrates with the inherent GAN loss, ensuring the integrity of pre-existing 3D-GAN architectures without necessitating alterations. We validate our approach with comprehensive experiments across various datasets and architectures, underscoring its efficacy. Our project website: https://3dgan-ssl.github.io
Code (0)
등록된 구현이 없습니다.
Tasks
Generative Adversarial NetworkSelf-Supervised LearningvalidSimilar Papers 제목 키워드 기반
Handling geometrical variability in nonlinear reduced order modeling through Continuous Geometry-Aware DL-ROMs
Deep Learning-based Reduced Order Models (DL-ROMs) provide nowadays a well-established class of accurate surrogate models for complex physical systems described by parametrized PDEs, by nonlinearly compressing the soluti…
Inductive BiasEnhancing self-supervised monocular depth estimation with traditional visual odometry
Estimating depth from a single image represents an attractive alternative to more traditional approaches leveraging multiple cameras. In this field, deep learning yielded outstanding results at the cost of needing large …
Depth And Camera MotionDepth EstimationMonocular Depth EstimationVisual OdometryGPA-VGGT:Adapting VGGT to Large Scale Localization by Self-Supervised Learning with Geometry and Physics Aware Loss
Transformer-based general visual geometry frameworks have shown promising performance in camera pose estimation and 3D scene understanding. Recent advancements in Visual Geometry Grounded Transformer (VGGT) models have s…
Self-Supervised LearningCamera Pose EstimationScene Understanding3D ReconstructionE-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training
Self-supervised pre-training has driven rapid progress in foundation models for language, 2D images, and video, yet remains largely unexplored for learning 3D-aware representations from multi-view images. In this paper, …
3D ReconstructionPose EstimationGAPS: Geometry-Aware, Physics-Based, Self-Supervised Neural Garment Draping
Recent neural, physics-based modeling of garment deformations allows faster and visually aesthetic results as opposed to the existing methods. Material-specific parameters are used by the formulation to control the garme…