Quota-based debiasing can decrease representation of already underrepresented groups
Many important decisions in societies such as school admissions, hiring, or elections are based on the selection of top-ranking individuals from a larger pool of candidates. This process is often subject to biases, which typically manifest as an under-representation of certain groups among the selected or accepted individuals. The most common approach to this issue is debiasing, for example via the introduction of quotas that ensure proportional representation of groups with respect to a certain, often binary attribute. Cases include quotas for women on corporate boards or ethnic quotas in elections. This, however, has the potential to induce changes in representation with respect to other attributes. For the case of two correlated binary attributes we show that quota-based debiasing based on a single attribute can worsen the representation of already underrepresented groups and decrease overall fairness of selection. We use several data sets from a broad range of domains from recidivism risk assessments to scientific citations to assess this effect in real-world settings. Our results demonstrate the importance of including all relevant attributes in debiasing procedures and that more efforts need to be put into eliminating the root causes of inequalities as purely numerical solutions such as quota-based debiasing might lead to unintended consequences.
Code (1)
Tasks
AttributeFairnessSimilar Papers 제목 키워드 기반
Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems
Representation bias is one of the most common types of biases in artificial intelligence (AI) systems, causing AI models to perform poorly on underrepresented data segments. Although AI practitioners use various methods …
Testing the Presence of Implicit Hiring Quotas with Application to German Universities
It is widely accepted that women are underrepresented in academia in general and economics in particular. This paper introduces a test to detect an under-researched form of hiring bias: implicit quotas. I derive a test u…
TARA: Training and Representation Alteration for AI Fairness and Domain Generalization
We propose a novel method for enforcing AI fairness with respect to protected or sensitive factors. This method uses a dual strategy performing training and representation alteration (TARA) for the mitigation of prominen…
Domain AdaptationDomain GeneralizationFairnessRepresentation LearningExplainable Disparity Compensation for Efficient Fair Ranking
Ranking functions that are used in decision systems often produce disparate results for different populations because of bias in the underlying data. Addressing, and compensating for, these disparate outcomes is a critic…
Decision MakingFairnessData Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selection
Machine learning models can fail on subgroups that are underrepresented during training. While techniques such as dataset balancing can improve performance on underperforming groups, they require access to training group…