A Self-Supervised Bootstrap Method for Single-Image 3D Face Reconstruction
State-of-the-art methods for 3D reconstruction of faces from a single image require 2D-3D pairs of ground-truth data for supervision. Such data is costly to acquire, and most datasets available in the literature are restricted to pairs for which the input 2D images depict faces in a near fronto-parallel pose. Therefore, many data-driven methods for single-image 3D facial reconstruction perform poorly on profile and near-profile faces. We propose a method to improve the performance of single-image 3D facial reconstruction networks by utilizing the network to synthesize its own training data for fine-tuning, comprising: (i) single-image 3D reconstruction of faces in near-frontal images without ground-truth 3D shape; (ii) application of a rigid-body transformation to the reconstructed face model; (iii) rendering of the face model from new viewpoints; and (iv) use of the rendered image and corresponding 3D reconstruction as additional data for supervised fine-tuning. The new 2D-3D pairs thus produced have the same high-quality observed for near fronto-parallel reconstructions, thereby nudging the network towards more uniform performance as a function of the viewing angle of input faces. Application of the proposed technique to the fine-tuning of a state-of-the-art single-image 3D-reconstruction network for faces demonstrates the usefulness of the method, with particularly significant gains for profile or near-profile views.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Face Reconstruction3D ReconstructionFace ModelFace ReconstructionSimilar Papers 제목 키워드 기반
CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrapping
Leveraging nearest neighbor retrieval for self-supervised representation learning has proven beneficial with object-centric images. However, this approach faces limitations when applied to scene-centric datasets, where m…
In-Context LearningObjectRepresentation LearningRetrieval+1InverseFaceNet: Deep Monocular Inverse Face Rendering
We introduce InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image. By estimating all par…
Face ReconstructionInverse RenderingBootstrapped Self-Supervised Training with Monocular Video for Semantic Segmentation and Depth Estimation
For a robot deployed in the world, it is desirable to have the ability of autonomous learning to improve its initial pre-set knowledge. We formalize this as a bootstrapped self-supervised learning problem where a system …
Depth EstimationSelf-Supervised LearningSemantic SegmentationAdaptive Similarity Bootstrapping for Self-Distillation based Representation Learning
Most self-supervised methods for representation learning leverage a cross-view consistency objective i.e., they maximize the representation similarity of a given image's augmented views. Recent work NNCLR goes beyond the…
Contrastive LearningRepresentation LearningS2F2: Self-Supervised High Fidelity Face Reconstruction from Monocular Image
We present a novel face reconstruction method capable of reconstructing detailed face geometry, spatially varying face reflectance from a single monocular image. We build our work upon the recent advances of DNN-based au…
3D Face ReconstructionFace ReconstructionSelf-Supervised LearningVocal Bursts Intensity Prediction