paper-with-me

홈 › Papers

Self-supervised learning of a facial attribute embedding from video

2018-08-21 · Olivia Wiles, A. Sophia Koepke, Andrew Zisserman

We propose a self-supervised framework for learning facial attributes by simply watching videos of a human face speaking, laughing, and moving over time. To perform this task, we introduce a network, Facial Attributes-Net (FAb-Net), that is trained to embed multiple frames from the same video face-track into a common low-dimensional space. With this approach, we make three contributions: first, we show that the network can leverage information from multiple source frames by predicting confidence/attention masks for each frame; second, we demonstrate that using a curriculum learning regime improves the learned embedding; finally, we demonstrate that the network learns a meaningful face embedding that encodes information about head pose, facial landmarks and facial expression, i.e. facial attributes, without having been supervised with any labelled data. We are comparable or superior to state-of-the-art self-supervised methods on these tasks and approach the performance of supervised methods.

📄 PDF Abstract BibTeX arXiv:1808.06882

Code (2)

jiarenchang/facecycle pytorch
oawiles/FAb-Net pytorch

Tasks

AttributeSelf-Supervised LearningUnsupervised Facial Landmark Detection

Similar Papers 제목 키워드 기반

MARLIN: Masked Autoencoder for facial video Representation LearnINg

2022-11-12 · CVPR 2023 1 · Zhixi Cai, Shreya Ghosh, Kalin Stefanov, Abhinav Dhall 외

This paper proposes a self-supervised approach to learn universal facial representations from videos, that can transfer across a variety of facial analysis tasks such as Facial Attribute Recognition (FAR), Facial Express…

Action ClassificationAttributeDeepFake DetectionEmotion Classification+8

Unsupervised Facial Performance Editing via Vector-Quantized StyleGAN Representations

2023-01-01 · ICCV 2023 1 · Berkay Kicanaoglu, Pablo Garrido, Gaurav Bharaj

High-fidelity virtual human avatar applications create a need for photorealistic video face synthesis with controllable semantic editing over facial features. While recent generative neural methods have shown signifi…

Face GenerationFace ModelFace ReenactmentQuantization+1

Towards an objective characterization of an individual's facial movements using Self-Supervised Person-Specific-Models

2022-11-15 · Yanis Tazi, Michael Berger, Winrich A. Freiwald

Disentangling facial movements from other facial characteristics, particularly from facial identity, remains a challenging task, as facial movements display great variation between individuals. In this paper, we aim to c…

Decoder

SVFAP: Self-supervised Video Facial Affect Perceiver

2023-12-31 · Licai Sun, Zheng Lian, Kexin Wang, Yu He 외

Video-based facial affect analysis has recently attracted increasing attention owing to its critical role in human-computer interaction. Previous studies mainly focus on developing various deep learning architectures and…

Dynamic Facial Expression RecognitionEmotion RecognitionFacial Expression RecognitionSelf-Supervised Learning

Contrastive Learning of Person-independent Representations for Facial Action Unit Detection

2024-03-06 · Yong Li, Shiguang Shan

Facial action unit (AU) detection, aiming to classify AU present in the facial image, has long suffered from insufficient AU annotations. In this paper, we aim to mitigate this data scarcity issue by learning AU represen…

Action Unit DetectionContrastive LearningFacial Action Unit DetectionRepresentation Learning