What Should Be Balanced in a "Balanced" Face Recognition Dataset?
The issue of demographic disparities in face recognition accuracy has attracted increasing attention in recent years. Various face image datasets have been proposed as 'fair' or 'balanced' to assess the accuracy of face recognition algorithms across demographics. These datasets typically balance the number of identities and images across demographics. It is important to note that the number of identities and images in an evaluation dataset are {\em not} driving factors for 1-to-1 face matching accuracy. Moreover, balancing the number of identities and images does not ensure balance in other factors known to impact accuracy, such as head pose, brightness, and image quality. We demonstrate these issues using several recently proposed datasets. To improve the ability to perform less biased evaluations, we propose a bias-aware toolkit that facilitates creation of cross-demographic evaluation datasets balanced on factors mentioned in this paper.
Code (1)
Tasks
Face RecognitionSimilar Papers 제목 키워드 기반
The Gender Gap in Face Recognition Accuracy Is a Hairy Problem
It is broadly accepted that there is a "gender gap" in face recognition accuracy, with females having higher false match and false non-match rates. However, relatively little is known about the cause(s) of this gender ga…
AttributeFace RecognitionBalanced Alignment for Face Recognition: A Joint Learning Approach
Face alignment is crucial for face recognition and has been widely adopted. However, current practice is too simple and under-explored. There lacks an understanding of how important face alignment is and how it should be…
Face AlignmentFace RecognitionBalanced Masked and Standard Face Recognition
We present the improved network architecture, data augmentation, and training strategies for the Webface track and Insightface/Glint360K track of the masked face recognition challenge of ICCV2021. One of the key goals is…
Data AugmentationFace DetectionFace RecognitionA Comparison of Human and Machine Learning Errors in Face Recognition
Machine learning applications in high-stakes scenarios should always operate under human oversight. Developing an optimal combination of human and machine intelligence requires an understanding of their complementarities…
Face RecognitionAdaptiveFace: Adaptive Margin and Sampling for Face Recognition
Training large-scale unbalanced data is the central topic in face recognition. In the past two years, face recognition has achieved remarkable improvements due to the introduction of margin based Softmax loss. However, t…
Face Recognition