Multimodal Conditional 3D Face Geometry Generation
We present a new method for multimodal conditional 3D face geometry generation that allows user-friendly control over the output identity and expression via a number of different conditioning signals. Within a single model, we demonstrate 3D faces generated from artistic sketches, 2D face landmarks, Canny edges, FLAME face model parameters, portrait photos, or text prompts. Our approach is based on a diffusion process that generates 3D geometry in a 2D parameterized UV domain. Geometry generation passes each conditioning signal through a set of cross-attention layers (IP-Adapter), one set for each user-defined conditioning signal. The result is an easy-to-use 3D face generation tool that produces high resolution geometry with fine-grain user control.
Code (0)
등록된 구현이 없습니다.
Tasks
3D geometryFace GenerationFace ModelMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator
Multimodal-driven talking face generation refers to animating a portrait with the given pose, expression, and gaze transferred from the driving image and video, or estimated from the text and audio. However, existing met…
DenoisingFace GenerationFace SwappingTalking Face GenerationVariational methods for Conditional Multimodal Deep Learning
In this paper, we address the problem of conditional modality learning, whereby one is interested in generating one modality given the other. While it is straightforward to learn a joint distribution over multiple modali…
Deep LearningMultimodal Deep LearningDeep Generative Methods and Tire Architecture Design
As deep generative models proliferate across the AI landscape, industrial practitioners still face critical yet unanswered questions about which deep generative models best suit complex manufacturing design tasks. This w…
Photo-Realistic Facial Details Synthesis from Single Image
We present a single-image 3D face synthesis technique that can handle challenging facial expressions while recovering fine geometric details. Our technique employs expression analysis for proxy face geometry generation a…
Face GenerationSpherical Geometry Diffusion: Generating High-quality 3D Face Geometry via Sphere-anchored Representations
A fundamental challenge in text-to-3D face generation is achieving high-quality geometry. The core difficulty lies in the arbitrary and intricate distribution of vertices in 3D space, making it challenging for existing m…
3D Generation