Text-Driven Diverse Facial Texture Generation via Progressive Latent-Space Refinement

2024-04-15 · Chi Wang, Junming Huang, Rong Zhang, Qi Wang, Haotian Yang, Haibin Huang, Chongyang Ma, Weiwei Xu

Automatic 3D facial texture generation has gained significant interest recently. Existing approaches may not support the traditional physically based rendering pipeline or rely on 3D data captured by Light Stage. Our key contribution is a progressive latent space refinement approach that can bootstrap from 3D Morphable Models (3DMMs)-based texture maps generated from facial images to generate high-quality and diverse PBR textures, including albedo, normal, and roughness. It starts with enhancing Generative Adversarial Networks (GANs) for text-guided and diverse texture generation. To this end, we design a self-supervised paradigm to overcome the reliance on ground truth 3D textures and train the generative model with only entangled texture maps. Besides, we foster mutual enhancement between GANs and Score Distillation Sampling (SDS). SDS boosts GANs with more generative modes, while GANs promote more efficient optimization of SDS. Furthermore, we introduce an edge-aware SDS for multi-view consistent facial structure. Experiments demonstrate that our method outperforms existing 3D texture generation methods regarding photo-realistic quality, diversity, and efficiency.

📄 PDF Abstract BibTeX arXiv:2404.09540

Code (0)

등록된 구현이 없습니다.

Tasks

DiversityTexture Synthesis

OMGTex: One-stage Multi-style Facial Texture Reconstruction without Geometry Guidance

2026-05-25 · Zitong Xiao, Yuda Qiu, Zisheng Ye, Xiaoguang Han arxiv

We propose OMGTex, an end-to-end diffusion-based framework for reconstructing high-quality and editable facial UV textures from multi-style facial images. Existing texture reconstruction methods face two major limitation…

Style Transfer

Geometry-Aware Texture Generation for 3D Head Modeling with Artist-driven Control

2025-05-07 · Amin Fadaeinejad, Abdallah Dib, Luiz Gustavo Hafemann, Emeline Got 외

Creating realistic 3D head assets for virtual characters that match a precise artistic vision remains labor-intensive. We present a novel framework that streamlines this process by providing artists with intuitive contro…

Texture Synthesis

Video-driven Neural Physically-based Facial Asset for Production

2022-02-11 · Longwen Zhang, Chuxiao Zeng, Qixuan Zhang, Hongyang Lin 외

Production-level workflows for producing convincing 3D dynamic human faces have long relied on an assortment of labor-intensive tools for geometry and texture generation, motion capture and rigging, and expression synthe…

4k motion retargeting Texture Synthesis

Unsupervised High-Fidelity Facial Texture Generation and Reconstruction

2021-10-10 · Ron Slossberg, Ibrahim Jubran, Ron Kimmel

Many methods have been proposed over the years to tackle the task of facial 3D geometry and texture recovery from a single image. Such methods often fail to provide high-fidelity texture without relying on 3D facial scan…

3D geometry Image Generation Texture Synthesis Transfer Learning+1

UV-IDM: Identity-Conditioned Latent Diffusion Model for Face UV-Texture Generation

2024-01-01 · CVPR 2024 1 · Hong Li, Yutang Feng, Song Xue, Xuhui Liu 외

3D face reconstruction aims at generating high-fidelity 3D face shapes and textures from single-view or multi-view images. However current prevailing facial texture generation methods generally suffer from low-qualit…

3D Face Reconstruction Face Model Face Reconstruction Texture Synthesis