Face De-occlusion using 3D Morphable Model and Generative Adversarial Network
In recent decades, 3D morphable model (3DMM) has been commonly used in image-based photorealistic 3D face reconstruction. However, face images are often corrupted by serious occlusion by non-face objects including eyeglasses, masks, and hands. Such objects block the correct capture of landmarks and shading information. Therefore, the reconstructed 3D face model is hardly reusable. In this paper, a novel method is proposed to restore de-occluded face images based on inverse use of 3DMM and generative adversarial network. We utilize the 3DMM prior to the proposed adversarial network and combine a global and local adversarial convolutional neural network to learn face de-occlusion model. The 3DMM serves not only as geometric prior but also proposes the face region for the local discriminator. Experiment results confirm the effectiveness and robustness of the proposed algorithm in removing challenging types of occlusions with various head poses and illumination. Furthermore, the proposed method reconstructs the correct 3D face model with de-occluded textures.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Face ReconstructionFace ModelFace ReconstructionGenerative Adversarial NetworkSimilar Papers 제목 키워드 기반
3D Morphable Models as Spatial Transformer Networks
In this paper, we show how a 3D Morphable Model (i.e. a statistical model of the 3D shape of a class of objects such as faces) can be used to spatially transform input data as a module (a 3DMM-STN) within a convolutional…
ClipFace: Text-guided Editing of Textured 3D Morphable Models
We propose ClipFace, a novel self-supervised approach for text-guided editing of textured 3D morphable model of faces. Specifically, we employ user-friendly language prompts to enable control of the expressions as well a…
Texture SynthesisVisibility Constrained Generative Model for Depth-based 3D Facial Pose Tracking
In this paper, we propose a generative framework that unifies depth-based 3D facial pose tracking and face model adaptation on-the-fly, in the unconstrained scenarios with heavy occlusions and arbitrary facial expression…
Face ModelPose EstimationPose TrackingTowards Large-Pose Face Frontalization in the Wild
Despite recent advances in face recognition using deep learning, severe accuracy drops are observed for large pose variations in unconstrained environments. Learning pose-invariant features is one solution, but needs exp…
3D ReconstructionFace RecognitionGenerative Adversarial NetworkSemi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model
We propose a novel end-to-end semi-supervised adversarial framework to generate photorealistic face images of new identities with wide ranges of expressions, poses, and illuminations conditioned by a 3D morphable model. …
Domain AdaptationFace GenerationFace RecognitionFace Verification+1