Hypothesis Engineering for Zero-Shot Hate Speech Detection
Standard approaches to hate speech detection rely on sufficient available hate speech annotations. Extending previous work that repurposes natural language inference (NLI) models for zero-shot text classification, we propose a simple approach that combines multiple hypotheses to improve English NLI-based zero-shot hate speech detection. We first conduct an error analysis for vanilla NLI-based zero-shot hate speech detection and then develop four strategies based on this analysis. The strategies use multiple hypotheses to predict various aspects of an input text and combine these predictions into a final verdict. We find that the zero-shot baseline used for the initial error analysis already outperforms commercial systems and fine-tuned BERT-based hate speech detection models on HateCheck. The combination of the proposed strategies further increases the zero-shot accuracy of 79.4% on HateCheck by 7.9 percentage points (pp), and the accuracy of 69.6% on ETHOS by 10.0pp.
Code (1)
Tasks
Hate Speech DetectionNatural Language InferenceText ClassificationZero-Shot Text ClassificationSimilar Papers 제목 키워드 기반
Exposing the limits of Zero-shot Cross-lingual Hate Speech Detection
Reducing and counter-acting hate speech on Social Media is a significant concern. Most of the proposed automatic methods are conducted exclusively on English and very few consistently labeled, non-English resources have …
Cross-Lingual TransferHate Speech DetectionTransfer LearningZero-Shot Cross-Lingual TransferMultilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models
Zero-shot cross-lingual transfer learning has been shown to be highly challenging for tasks involving a lot of linguistic specificities or when a cultural gap is present between languages, such as in hate speech detectio…
Cross-Lingual TransferHate Speech Detectionnamed-entity-recognitionNamed Entity Recognition+4Detecting Hate Speech with GPT-3
Sophisticated language models such as OpenAI's GPT-3 can generate hateful text that targets marginalized groups. Given this capacity, we are interested in whether large language models can be used to identify hate speech…
Few-Shot LearningHate Speech DetectionOne-Shot LearningCan Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study
Despite growing interest in automated hate speech detection, most existing approaches overlook the linguistic diversity of online content. Multilingual instruction-tuned large language models such as LLaMA, Aya, Qwen, an…
DiversityHate Speech DetectionWhen More is not Necessary Better: Multilingual Auxiliary Tasks for Zero-Shot Cross-Lingual Transfer of Hate Speech Detection Models
Zero-shot cross-lingual transfer learning has been shown to be highly challenging for tasks involving a lot of linguistic specificities or when a cultural gap is present between languages, such as in hate speech detectio…
Cross-Lingual TransferHate Speech DetectionLanguage ModelingLanguage Modelling+6