BiasX: "Thinking Slow" in Toxic Content Moderation with Explanations of Implied Social Biases
Toxicity annotators and content moderators often default to mental shortcuts when making decisions. This can lead to subtle toxicity being missed, and seemingly toxic but harmless content being over-detected. We introduce BiasX, a framework that enhances content moderation setups with free-text explanations of statements' implied social biases, and explore its effectiveness through a large-scale crowdsourced user study. We show that indeed, participants substantially benefit from explanations for correctly identifying subtly (non-)toxic content. The quality of explanations is critical: imperfect machine-generated explanations (+2.4% on hard toxic examples) help less compared to expert-written human explanations (+7.2%). Our results showcase the promise of using free-text explanations to encourage more thoughtful toxicity moderation.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Watch Your Language: Investigating Content Moderation with Large Language Models
Large language models (LLMs) have exploded in popularity due to their ability to perform a wide array of natural language tasks. Text-based content moderation is one LLM use case that has received recent enthusiasm, howe…
Validating Multimedia Content Moderation Software via Semantic Fusion
The exponential growth of social media platforms, such as Facebook and TikTok, has revolutionized communication and content publication in human society. Users on these platforms can publish multimedia content that deliv…
Sentencesoftware testingAn Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software
The exponential growth of social media platforms has brought about a revolution in communication and content dissemination in human society. Nevertheless, these platforms are being increasingly misused to spread toxic co…
MTTM: Metamorphic Testing for Textual Content Moderation Software
The exponential growth of social media platforms such as Twitter and Facebook has revolutionized textual communication and textual content publication in human society. However, they have been increasingly exploited to p…
SentenceTiny-Toxic-Detector: A compact transformer-based model for toxic content detection
This paper presents Tiny-toxic-detector, a compact transformer-based model designed for toxic content detection. Despite having only 2.1 million parameters, Tiny-toxic-detector achieves competitive performance on benchma…
Computational Efficiency