How Gender Debiasing Affects Internal Model Representations, and Why It Matters
Common studies of gender bias in NLP focus either on extrinsic bias measured by model performance on a downstream task or on intrinsic bias found in models' internal representations. However, the relationship between extrinsic and intrinsic bias is relatively unknown. In this work, we illuminate this relationship by measuring both quantities together: we debias a model during downstream fine-tuning, which reduces extrinsic bias, and measure the effect on intrinsic bias, which is operationalized as bias extractability with information-theoretic probing. Through experiments on two tasks and multiple bias metrics, we show that our intrinsic bias metric is a better indicator of debiasing than (a contextual adaptation of) the standard WEAT metric, and can also expose cases of superficial debiasing. Our framework provides a comprehensive perspective on bias in NLP models, which can be applied to deploy NLP systems in a more informed manner. Our code and model checkpoints are publicly available.
Code (2)
Similar Papers 제목 키워드 기반
How Gender Debiasing Affects Internal Model Representations, and Why It Matters
Common studies of gender bias in NLP focus either on extrinsic bias measured by model performance on a downstream task or on intrinsic bias found in models' internal representations. However, the relationship between ext…
How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?
Many natural languages assign grammatical gender also to inanimate nouns in the language. In such languages, words that relate to the gender-marked nouns are inflected to agree with the noun's gender. We show that this a…
Word EmbeddingsGender Encoding Patterns in Pretrained Language Model Representations
Gender bias in pretrained language models (PLMs) poses significant social and ethical challenges. Despite growing awareness, there is a lack of comprehensive investigation into how different models internally represent a…
Language ModelingLanguage ModellingmodelDebiasing Embeddings for Reduced Gender Bias in Text Classification
(Bolukbasi et al., 2016) demonstrated that pretrained word embeddings can inherit gender bias from the data they were trained on. We investigate how this bias affects downstream classification tasks, using the case study…
ClassificationGeneral Classificationtext-classificationText Classification+1Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search
Internet search affects people's cognition of the world, so mitigating biases in search results and learning fair models is imperative for social good. We study a unique gender bias in image search in this work: the sear…
Image RetrievalNatural Language Queries