Self-supervised learning of a facial attribute embedding from video
We propose a self-supervised framework for learning facial attributes by simply watching videos of a human face speaking, laughing, and moving over time. To perform this task, we introduce a network, Facial Attributes-Net (FAb-Net), that is trained to embed multiple frames from the same video face-track into a common low-dimensional space. With this approach, we make three contributions: first, we show that the network can leverage information from multiple source frames by predicting confidence/attention masks for each frame; second, we demonstrate that using a curriculum learning regime improves the learned embedding; finally, we demonstrate that the network learns a meaningful face embedding that encodes information about head pose, facial landmarks and facial expression, i.e. facial attributes, without having been supervised with any labelled data. We are comparable or superior to state-of-the-art self-supervised methods on these tasks and approach the performance of supervised methods.
Code (2)
Tasks
AttributeSelf-Supervised LearningUnsupervised Facial Landmark DetectionSimilar Papers 제목 키워드 기반
MARLIN: Masked Autoencoder for facial video Representation LearnINg
This paper proposes a self-supervised approach to learn universal facial representations from videos, that can transfer across a variety of facial analysis tasks such as Facial Attribute Recognition (FAR), Facial Express…
Action ClassificationAttributeDeepFake DetectionEmotion Classification+8Unsupervised Facial Performance Editing via Vector-Quantized StyleGAN Representations
High-fidelity virtual human avatar applications create a need for photorealistic video face synthesis with controllable semantic editing over facial features. While recent generative neural methods have shown signifi…
Face GenerationFace ModelFace ReenactmentQuantization+1Towards an objective characterization of an individual's facial movements using Self-Supervised Person-Specific-Models
Disentangling facial movements from other facial characteristics, particularly from facial identity, remains a challenging task, as facial movements display great variation between individuals. In this paper, we aim to c…
DecoderSVFAP: Self-supervised Video Facial Affect Perceiver
Video-based facial affect analysis has recently attracted increasing attention owing to its critical role in human-computer interaction. Previous studies mainly focus on developing various deep learning architectures and…
Dynamic Facial Expression RecognitionEmotion RecognitionFacial Expression RecognitionSelf-Supervised LearningContrastive Learning of Person-independent Representations for Facial Action Unit Detection
Facial action unit (AU) detection, aiming to classify AU present in the facial image, has long suffered from insufficient AU annotations. In this paper, we aim to mitigate this data scarcity issue by learning AU represen…
Action Unit DetectionContrastive LearningFacial Action Unit DetectionRepresentation Learning