Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
We introduce a new benchmark, WinoBias, for coreference resolution focused on gender bias. Our corpus contains Winograd-schema style sentences with entities corresponding to people referred by their occupation (e.g. the nurse, the doctor, the carpenter). We demonstrate that a rule-based, a feature-rich, and a neural coreference system all link gendered pronouns to pro-stereotypical entities with higher accuracy than anti-stereotypical entities, by an average difference of 21.1 in F1 score. Finally, we demonstrate a data-augmentation approach that, in combination with existing word-embedding debiasing techniques, removes the bias demonstrated by these systems in WinoBias without significantly affecting their performance on existing coreference benchmark datasets. Our dataset and code are available at http://winobias.org.
Code (4)
Tasks
coreference-resolutionCoreference ResolutionData AugmentationSimilar Papers 제목 키워드 기반
Transforming Dutch: Debiasing Dutch Coreference Resolution Systems for Non-binary Pronouns
Gender-neutral pronouns are increasingly being introduced across Western languages. Recent evaluations have however demonstrated that English NLP systems are unable to correctly process gender-neutral pronouns, with the …
coreference-resolutionCoreference ResolutioncounterfactualData AugmentationSecond Order WinoBias (SoWinoBias) Test Set for Latent Gender Bias Detection in Coreference Resolution
We observe an instance of gender-induced bias in a downstream application, despite the absence of explicit gender words in the test cases. We provide a test set, SoWinoBias, for the purpose of measuring such latent gende…
Bias Detectioncoreference-resolutionCoreference ResolutionGender Bias DetectionA Causal Inference Method for Reducing Gender Bias in Word Embedding Relations
Word embedding has become essential for natural language processing as it boosts empirical performances of various tasks. However, recent research discovers that gender bias is incorporated in neural word embeddings, and…
Causal Inferencecoreference-resolutionCoreference ResolutionSentence+1Nurse is Closer to Woman than Surgeon? Mitigating Gender-Biased Proximities in Word Embeddings
Word embeddings are the standard model for semantic and syntactic representations of words. Unfortunately, these models have been shown to exhibit undesirable word associations resulting from gender, racial, and religiou…
coreference-resolutionCoreference ResolutionWord EmbeddingsGender Bias in Coreference Resolution
We present an empirical study of gender bias in coreference resolution systems. We first introduce a novel, Winograd schema-style set of minimal pair sentences that differ only by pronoun gender. With these "Winogender s…
coreference-resolutionCoreference Resolution