The Impact of Unstated Norms in Bias Analysis of Language Models
Bias in large language models (LLMs) has many forms, from overt discrimination to implicit stereotypes. Counterfactual bias evaluation is a widely used approach to quantifying bias and often relies on template-based probes that explicitly state group membership. It measures whether the outcome of a task performed by an LLM is invariant to a change in group membership. In this work, we find that template-based probes can lead to unrealistic bias measurements. For example, LLMs appear to mistakenly cast text associated with White race as negative at higher rates than other groups. We hypothesize that this arises artificially via a mismatch between commonly unstated norms, in the form of markedness, in the pretraining text of LLMs (e.g., Black president vs. president) and templates used for bias measurement (e.g., Black president vs. White president). The findings highlight the potential misleading impact of varying group membership through explicit mention in counterfactual bias quantification.
Code (0)
등록된 구현이 없습니다.
Tasks
Bias DetectioncounterfactualSentiment AnalysisSimilar Papers 제목 키워드 기반
This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Models
Bias research in NLP seeks to analyse models for social biases, thus helping NLP practitioners uncover, measure, and mitigate social harms. We analyse the body of work that uses prompts and templates to assess bias in la…
Experimental DesignExamining Temporal Bias in Abusive Language Detection
The use of abusive language online has become an increasingly pervasive problem that damages both individuals and society, with effects ranging from psychological harm right through to escalation to real-life violence an…
Abusive LanguageKnowledge of cultural moral norms in large language models
Moral norms vary across cultures. A recent line of work suggests that English large language models contain human-like moral biases, but these studies typically do not examine moral variation in a diverse cultural settin…
DiversitySurveyBias Mitigation or Cultural Commonsense? Evaluating LLMs with a Japanese Dataset
Large language models (LLMs) exhibit social biases, prompting the development of various debiasing methods. However, debiasing methods may degrade the capabilities of LLMs. Previous research has evaluated the impact of b…
Speaking Multiple Languages Affects the Moral Bias of Language Models
Pre-trained multilingual language models (PMLMs) are commonly used when dealing with data from multiple languages and cross-lingual transfer. However, PMLMs are trained on varying amounts of data for each language. In pr…
Cross-Lingual Transfer