Leveraging Synthetic Data for Generalizable and Fair Facial Action Unit Detection
Facial action unit (AU) detection is a fundamental block for objective facial expression analysis. Supervised learning approaches require a large amount of manual labeling which is costly. The limited labeled data are also not diverse in terms of gender which can affect model fairness. In this paper, we propose to use synthetically generated data and multi-source domain adaptation (MSDA) to address the problems of the scarcity of labeled data and the diversity of subjects. Specifically, we propose to generate a diverse dataset through synthetic facial expression re-targeting by transferring the expressions from real faces to synthetic avatars. Then, we use MSDA to transfer the AU detection knowledge from a real dataset and the synthetic dataset to a target dataset. Instead of aligning the overall distributions of different domains, we propose Paired Moment Matching (PM2) to align the features of the paired real and synthetic data with the same facial expression. To further improve gender fairness, PM2 matches the features of the real data with a female and a male synthetic image. Our results indicate that synthetic data and the proposed model improve both AU detection performance and fairness across genders, demonstrating its potential to solve AU detection in-the-wild.
Code (0)
등록된 구현이 없습니다.
Tasks
Action Unit DetectionDomain AdaptationFacial Action Unit DetectionFairnessMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Self-Supervised Learning Pipeline for Demographically Fair Facial Attribute Classification
Published research highlights the presence of demographic bias in automated facial attribute classification. The proposed bias mitigation techniques are mostly based on supervised learning, which requires a large amount …
AttributeContrastive LearningFacial Attribute ClassificationFairness+4Fast & Fair: Efficient Second-Order Robust Optimization for Fairness in Machine Learning
This project explores adversarial training techniques to develop fairer Deep Neural Networks (DNNs) to mitigate the inherent bias they are known to exhibit. DNNs are susceptible to inheriting bias with respect to sensiti…
FairnessQuality-Diversity Generative Sampling for Learning with Synthetic Data
Generative models can serve as surrogates for some real data sources by creating synthetic training datasets, but in doing so they may transfer biases to downstream tasks. We focus on protecting quality and diversity whe…
DiversityFairnessFineFACE: Fair Facial Attribute Classification Leveraging Fine-grained Features
Published research highlights the presence of demographic bias in automated facial attribute classification algorithms, particularly impacting women and individuals with darker skin tones. Existing bias mitigation techni…
AttributeClassificationFacial Attribute ClassificationFairnessUnlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection
This paper tackles the challenge of detecting partially manipulated facial deepfakes, which involve subtle alterations to specific facial features while retaining the overall context, posing a greater detection difficult…
DeepFake DetectionFace SwappingMetric Learningparameter-efficient fine-tuning