The Lou Dataset -- Exploring the Impact of Gender-Fair Language in German Text Classification
Gender-fair language, an evolving German linguistic variation, fosters inclusion by addressing all genders or using neutral forms. Nevertheless, there is a significant lack of resources to assess the impact of this linguistic shift on classification using language models (LMs), which are probably not trained on such variations. To address this gap, we present Lou, the first dataset featuring high-quality reformulations for German text classification covering seven tasks, like stance detection and toxicity classification. Evaluating 16 mono- and multi-lingual LMs on Lou shows that gender-fair language substantially impacts predictions by flipping labels, reducing certainty, and altering attention patterns. However, existing evaluations remain valid, as LM rankings of original and reformulated instances do not significantly differ. While we offer initial insights on the effect on German text classification, the findings likely apply to other languages, as consistent patterns were observed in multi-lingual and English LMs.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationStance Detectiontext-classificationText ClassificationvalidMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DiFair: A Benchmark for Disentangled Assessment of Gender Knowledge and Bias
Numerous debiasing techniques have been proposed to mitigate the gender bias that is prevalent in pretrained language models. These are often evaluated on datasets that check the extent to which the model is gender-neutr…
Language ModelingLanguage ModellingMasked Language ModelingFairness in Biometrics: a figure of merit to assess biometric verification systems
Machine learning-based (ML) systems are being largely deployed since the last decade in a myriad of scenarios impacting several instances in our daily lives. With this vast sort of applications, aspects of fairness start…
FairnessEmo-bias: A Large Scale Evaluation of Social Bias on Speech Emotion Recognition
The rapid growth of Speech Emotion Recognition (SER) has diverse global applications, from improving human-computer interactions to aiding mental health diagnostics. However, SER models might contain social bias toward g…
Emotion RecognitionSelf-Supervised LearningSpeech Emotion RecognitionExploring Gender Disparities in Automatic Speech Recognition Technology
This study investigates factors influencing Automatic Speech Recognition (ASR) systems' fairness and performance across genders, beyond the conventional examination of demographics. Using the LibriSpeech dataset and the …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Fairnessspeech-recognition+1GFG -- Gender-Fair Generation: A CALAMITA Challenge
Gender-fair language aims at promoting gender equality by using terms and expressions that include all identities and avoid reinforcing gender stereotypes. Implementing gender-fair strategies is particularly challenging …
Translation