HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping
In this work, we propose a high fidelity face swapping method, called HifiFace, which can well preserve the face shape of the source face and generate photo-realistic results. Unlike other existing face swapping works that only use face recognition model to keep the identity similarity, we propose 3D shape-aware identity to control the face shape with the geometric supervision from 3DMM and 3D face reconstruction method. Meanwhile, we introduce the Semantic Facial Fusion module to optimize the combination of encoder and decoder features and make adaptive blending, which makes the results more photo-realistic. Extensive experiments on faces in the wild demonstrate that our method can preserve better identity, especially on the face shape, and can generate more photo-realistic results than previous state-of-the-art methods.
Code (1)
Tasks
3D Face ReconstructionDecoderFace RecognitionFace ReconstructionFace SwappingVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Deep Octree-based CNNs with Output-Guided Skip Connections for 3D Shape and Scene Completion
Acquiring complete and clean 3D shape and scene data is challenging due to geometric occlusion and insufficient views during 3D capturing. We present a simple yet effective deep learning approach for completing the input…
Mixture-of-Shape-Experts (MoSE): End-to-End Shape Dictionary Framework to Prompt SAM for Generalizable Medical Segmentation
Single domain generalization (SDG) has recently attracted growing attention in medical image segmentation. One promising strategy for SDG is to leverage consistent semantic shape priors across different imaging protocols…
Dictionary LearningDomain GeneralizationImage SegmentationMedical Image Segmentation+22D Semantic-Guided Semantic Scene Completion
Semantic scene completion (SSC) aims to simultaneously perform scene completion (SC) and predict semantic categories of a 3D scene from a single depth and/or RGB image. Most existing SSC methods struggle to handle compl…
2D Semantic Segmentation3D Semantic Scene CompletionMissing ValuesSemantic SegmentationShapeLib: Designing a library of programmatic 3D shape abstractions with Large Language Models
We present ShapeLib, the first method that leverages the priors of LLMs to design libraries of programmatic 3D shape abstractions. Our system accepts two forms of design intent: text descriptions of functions to include …
Synthetic Data GenerationLLM-Guided Material Inference for 3D Point Clouds
Most existing 3D shape datasets and models focus solely on geometry, overlooking the material properties that determine how objects appear. We introduce a two-stage large language model (LLM) based method for inferring m…
Point Clouds