Nonlinear 3D Face Morphable Model

2018-04-11 · CVPR 2018 6 · Luan Tran, Xiaoming Liu

As a classic statistical model of 3D facial shape and texture, 3D Morphable Model (3DMM) is widely used in facial analysis, e.g., model fitting, image synthesis. Conventional 3DMM is learned from a set of well-controlled 2D face images with associated 3D face scans, and represented by two sets of PCA basis functions. Due to the type and amount of training data, as well as the linear bases, the representation power of 3DMM can be limited. To address these problems, this paper proposes an innovative framework to learn a nonlinear 3DMM model from a large set of unconstrained face images, without collecting 3D face scans. Specifically, given a face image as input, a network encoder estimates the projection, shape and texture parameters. Two decoders serve as the nonlinear 3DMM to map from the shape and texture parameters to the 3D shape and texture, respectively. With the projection parameter, 3D shape, and texture, a novel analytically-differentiable rendering layer is designed to reconstruct the original input face. The entire network is end-to-end trainable with only weak supervision. We demonstrate the superior representation power of our nonlinear 3DMM over its linear counterpart, and its contribution to face alignment and 3D reconstruction.

📄 PDF Abstract BibTeX arXiv:1804.03786

Code (1)

tranluan/Nonlinear_Face_3DMM 공식 구현 tf

Tasks

3D Reconstruction Face Alignment Image Generationmodel

Methods 이 논문이 사용한 방법론

PCA Principle Components Analysis (PCA) is an unsupervised method primary used for dimensionality reduction within machine learning. PCA is calculated via a singular value…

Towards High-fidelity Nonlinear 3D Face Morphable Model

2019-04-09 · CVPR 2019 6 · Luan Tran, Feng Liu, Xiaoming Liu

Embedding 3D morphable basis functions into deep neural networks opens great potential for models with better representation power. However, to faithfully learn those models from an image collection, it requires strong r…

3D Face Reconstruction Face Reconstruction model Vocal Bursts Intensity Prediction

Fitting a 3D Morphable Model to Edges: A Comparison Between Hard and Soft Correspondences

2016-02-02 · Anil Bas, William A. P. Smith, Timo Bolkart, Stefanie Wuhrer

We propose a fully automatic method for fitting a 3D morphable model to single face images in arbitrary pose and lighting. Our approach relies on geometric features (edges and landmarks) and, inspired by the iterated clo…

On Learning 3D Face Morphable Model from In-the-wild Images

2018-08-28 · Luan Tran, Xiaoming Liu

As a classic statistical model of 3D facial shape and albedo, 3D Morphable Model (3DMM) is widely used in facial analysis, e.g., model fitting, image synthesis. Conventional 3DMM is learned from a set of 3D face scans wi…

3D Reconstruction Face Alignment Image Generation

ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations

2022-03-28 · CVPR 2022 1 · Mingwu Zheng, Hongyu Yang, Di Huang, Liming Chen

Precise representations of 3D faces are beneficial to various computer vision and graphics applications. Due to the data discretization and model linearity, however, it remains challenging to capture accurate identity an…

Face Alignment Face Model

Uncertainty-Aware Mesh Decoder for High Fidelity 3D Face Reconstruction

2020-06-01 · CVPR 2020 6 · Gun-Hee Lee, Seong-Whan Lee

3D Morphable Model (3DMM) is a statistical model of facial shape and texture using a set of linear basis functions. Most of the recent 3D face reconstruction methods aim to embed the 3D morphable basis functions into Dee…

3D Face Reconstruction Decoder Face Reconstruction Generative Adversarial Network+2