BioFaceNet: Deep Biophysical Face Image Interpretation
In this paper we present BioFaceNet, a deep CNN that learns to decompose a single face image into biophysical parameters maps, diffuse and specular shading maps as well as estimating the spectral power distribution of the scene illuminant and the spectral sensitivity of the camera. The network comprises a fully convolutional encoder for estimating the spatial maps with a fully connected branch for estimating the vector quantities. The network is trained using a self-supervised appearance loss computed via a model-based decoder. The task is highly underconstrained so we impose a number of model-based priors. Skin spectral reflectance is restricted to a biophysical model, we impose a statistical prior on camera spectral sensitivities, a physical constraint on illumination spectra, a sparsity prior on specular reflections and direct supervision on diffuse shading using a rough shape proxy. We show convincing qualitative results on in-the-wild data and introduce a benchmark for quantitative evaluation on this new task.
Code (3)
Tasks
DecoderSimilar Papers 제목 키워드 기반
Decomposing multispectral face images into diffuse and specular shading and biophysical parameters
We propose a novel biophysical and dichromatic reflectance model that efficiently characterises spectral skin reflectance. We show how to fit the model to multispectral face images enabling high quality estimation of dif…
The Sixth Law of Stupidity: A Biophysical Interpretation of Carlo Cipolla's Stupidity Laws
Carlo Cipolla's stupidity quadrant and his five laws of stupidity were proposed for the first time in 1976. Exposed in a humorous mood by the author, these concepts nevertheless describe very serious features of the inte…
Biophysical models of cis-regulation as interpretable neural networks
The adoption of deep learning techniques in genomics has been hindered by the difficulty of mechanistically interpreting the models that these techniques produce. In recent years, a variety of post-hoc attribution method…
Deep LearningPredicting variation of DNA shape preferences in protein-DNA interaction in cancer cells with a new biophysical model
DNA shape readout is an important mechanism of target site recognition by transcription factors, in addition to the sequence readout. Several models of transcription factor-DNA binding which consider DNA shape have been …
Estimation of Spectral Biophysical Skin Properties from Captured RGB Albedo
We present a new method to reconstruct and manipulate the spectral properties of human skin from simple RGB albedo captures. To this end, we leverage Monte Carlo light simulation over an accurate biophysical human skin l…