Towards Conceptualization of "Fair Explanation": Disparate Impacts of anti-Asian Hate Speech Explanations on Content Moderators
Recent research at the intersection of AI explainability and fairness has focused on how explanations can improve human-plus-AI task performance as assessed by fairness measures. We propose to characterize what constitutes an explanation that is itself "fair" -- an explanation that does not adversely impact specific populations. We formulate a novel evaluation method of "fair explanations" using not just accuracy and label time, but also psychological impact of explanations on different user groups across many metrics (mental discomfort, stereotype activation, and perceived workload). We apply this method in the context of content moderation of potential hate speech, and its differential impact on Asian vs. non-Asian proxy moderators, across explanation approaches (saliency map and counterfactual explanation). We find that saliency maps generally perform better and show less evidence of disparate impact (group) and individual unfairness than counterfactual explanations. Content warning: This paper contains examples of hate speech and racially discriminatory language. The authors do not support such content. Please consider your risk of discomfort carefully before continuing reading!
Code (1)
Tasks
counterfactualCounterfactual ExplanationFairnessSimilar Papers 제목 키워드 기반
Post-processing of Differentially Private Data: A Fairness Perspective
Post-processing immunity is a fundamental property of differential privacy: it enables arbitrary data-independent transformations to differentially private outputs without affecting their privacy guarantees. Post-process…
FairnessExplaining How Quantization Disparately Skews a Model
Post Training Quantization (PTQ) is widely adopted due to its high compression capacity and speed with minimal impact on accuracy. However, we observed that disparate impacts are exacerbated by quantization, especially f…
FairPut: A Light Framework for Machine Learning Fairness with LightGBM
This is a holistic framework to approach fair prediction outputs at the individual and group level. This framework includes quantitative monotonic measures, residual explanations, benchmark competition, adversarial attac…
BIG-bench Machine LearningFairnessThe Disparate Impacts of Speculative Decoding
The practice of speculative decoding, whereby inference is probabilistically supported by a smaller, cheaper, ``drafter'' model, has become a standard technique for systematically reducing the decoding time of large lang…
Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacy
Deployment of deep learning in different fields and industries is growing day by day due to its performance, which relies on the availability of data and compute. Data is often crowd-sourced and contains sensitive inform…
Fairness