Improving Fairness in Large-Scale Object Recognition by CrowdSourced Demographic Information
There has been increasing awareness of ethical issues in machine learning, and fairness has become an important research topic. Most fairness efforts in computer vision have been focused on human sensing applications and preventing discrimination by people's physical attributes such as race, skin color or age by increasing visual representation for particular demographic groups. We argue that ML fairness efforts should extend to object recognition as well. Buildings, artwork, food and clothing are examples of the objects that define human culture. Representing these objects fairly in machine learning datasets will lead to models that are less biased towards a particular culture and more inclusive of different traditions and values. There exist many research datasets for object recognition, but they have not carefully considered which classes should be included, or how much training data should be collected per class. To address this, we propose a simple and general approach, based on crowdsourcing the demographic composition of the contributors: we define fair relevance scores, estimate them, and assign them to each class. We showcase its application to the landmark recognition domain, presenting a detailed analysis and the final fairer landmark rankings. We present analysis which leads to a much fairer coverage of the world compared to existing datasets. The evaluation dataset was used for the 2021 Google Landmark Challenges, which was the first of a kind with an emphasis on fairness in generic object recognition.
Code (0)
등록된 구현이 없습니다.
Tasks
Cultural Vocal Bursts Intensity PredictionFairnessLandmark RecognitionObject RecognitionSimilar Papers 제목 키워드 기반
Optimal Fair Aggregation of Crowdsourced Noisy Labels using Demographic Parity Constraints
As acquiring reliable ground-truth labels is usually costly, or infeasible, crowdsourcing and aggregation of noisy human annotations is the typical resort. Aggregating subjective labels, though, may amplify individual bi…
Diversified Ensembling: An Experiment in Crowdsourced Machine Learning
Crowdsourced machine learning on competition platforms such as Kaggle is a popular and often effective method for generating accurate models. Typically, teams vie for the most accurate model, as measured by overall error…
FairnessHoldout Setimage-classificationImage ClassificationBayesian Detector Combination for Object Detection with Crowdsourced Annotations
Acquiring fine-grained object detection annotations in unconstrained images is time-consuming, expensive, and prone to noise, especially in crowdsourcing scenarios. Most prior object detection methods assume accurate ann…
Objectobject-detectionObject DetectionCrowdsourcing Learning as Domain Adaptation: A Case Study on Named Entity Recognition
Crowdsourcing is regarded as one prospective solution for effective supervised learning, aiming to build large-scale annotated training data by crowd workers. Previous studies focus on reducing the influences from the no…
Domain Adaptationnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2Improving Fairness in Speaker Recognition
The human voice conveys unique characteristics of an individual, making voice biometrics a key technology for verifying identities in various industries. Despite the impressive progress of speaker recognition systems in …
AttributeFairnessSpeaker Recognition