Interpretable Assessment of Fairness During Model Evaluation
For companies developing products or algorithms, it is important to understand the potential effects not only globally, but also on sub-populations of users. In particular, it is important to detect if there are certain groups of users that are impacted differently compared to others with regard to business metrics or for whom a model treats unequally along fairness concerns. In this paper, we introduce a novel hierarchical clustering algorithm to detect heterogeneity among users in given sets of sub-populations with respect to any specified notion of group similarity. We prove statistical guarantees about the output and provide interpretable results. We demonstrate the performance of the algorithm on real data from LinkedIn.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringFairnessmodelSimilar Papers 제목 키워드 기반
When to Invoke: Refining LLM Fairness with Toxicity Assessment
Large Language Models (LLMs) are increasingly used for toxicity assessment in online moderation systems, where fairness across demographic groups is essential for equitable treatment. However, LLMs often produce inconsis…
Fairness Index Measures to Evaluate Bias in Biometric Recognition
The demographic disparity of biometric systems has led to serious concerns regarding their societal impact as well as applicability of such systems in private and public domains. A quantitative evaluation of demographic …
BenchmarkingFairnessWhen Fairness Metrics Disagree: Evaluating the Reliability of Demographic Fairness Assessment in Machine Learning
The evaluation of fairness in machine learning systems has become a central concern in high-stakes applications, including biometric recognition, healthcare decision-making, and automated risk assessment. Existing approa…
Face RecognitionIn Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Prediction
Objectives: We study interpretable recidivism prediction using machine learning (ML) models and analyze performance in terms of prediction ability, sparsity, and fairness. Unlike previous works, this study trains interpr…
BIG-bench Machine LearningFairnessInterpretable Machine LearningWhich Demographic Features Are Relevant for Individual Fairness Evaluation of U.S. Recidivism Risk Assessment Tools?
Despite its constitutional relevance, the technical ``individual fairness'' criterion has not been operationalized in U.S. state or federal statutes/regulations. We conduct a human subjects experiment to address this gap…
Fairness