Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers
AI Safety Moderation (ASM) classifiers are designed to moderate content on social media platforms and to serve as guardrails that prevent Large Language Models (LLMs) from being fine-tuned on unsafe inputs. Owing to their potential for disparate impact, it is crucial to ensure that these classifiers: (1) do not unfairly classify content belonging to users from minority groups as unsafe compared to those from majority groups and (2) that their behavior remains robust and consistent across similar inputs. In this work, we thus examine the fairness and robustness of four widely-used, closed-source ASM classifiers: OpenAI Moderation API, Perspective API, Google Cloud Natural Language (GCNL) API, and Clarifai API. We assess fairness using metrics such as demographic parity and conditional statistical parity, comparing their performance against ASM models and a fair-only baseline. Additionally, we analyze robustness by testing the classifiers' sensitivity to small and natural input perturbations. Our findings reveal potential fairness and robustness gaps, highlighting the need to mitigate these issues in future versions of these models.
Code (1)
Tasks
FairnessSimilar Papers 제목 키워드 기반
Runtime Monitoring of Dynamic Fairness Properties
A machine-learned system that is fair in static decision-making tasks may have biased societal impacts in the long-run. This may happen when the system interacts with humans and feedback patterns emerge, reinforcing old …
Decision MakingFairnessMoral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues
As large language models take on morally consequential roles in healthcare, legal, and hiring contexts, we need to examine whether their ethical behaviors are genuine or superficial. We show that current fairness evaluat…
FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Models
Recent advancements in Large Language Models (LLMs) have significantly enhanced interactions between users and models. These advancements concurrently underscore the need for rigorous safety evaluations due to the manife…
FairnessFARMUR: Fair Adversarial Retraining to Mitigate Unfairness in Robustness
Deep Neural Networks (DNNs) have been deployed in safetycritical real-world applications, including automated decision-making systems. There are often concerns about two aspects of these systems: the fairness of the pred…
Decision MakingFairnessaiXamine: Simplified LLM Safety and Security
Evaluating Large Language Models (LLMs) for safety and security remains a complex task, often requiring users to navigate a fragmented landscape of ad hoc benchmarks, datasets, metrics, and reporting formats. To address …
2kAdversarial RobustnessFairnessHallucination+2