Measuring and Reducing Gendered Correlations in Pre-trained Models
Pre-trained models have revolutionized natural language understanding. However, researchers have found they can encode artifacts undesired in many applications, such as professions correlating with one gender more than another. We explore such gendered correlations as a case study for how to address unintended correlations in pre-trained models. We define metrics and reveal that it is possible for models with similar accuracy to encode correlations at very different rates. We show how measured correlations can be reduced with general-purpose techniques, and highlight the trade offs different strategies have. With these results, we make recommendations for training robust models: (1) carefully evaluate unintended correlations, (2) be mindful of seemingly innocuous configuration differences, and (3) focus on general mitigations.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language UnderstandingSimilar Papers 제목 키워드 기반
Gendered Language in Resumes
Despite growing concerns around gender bias in NLP models used in algorithmic hiring, there is little empirical work studying the extent and nature of gendered language in resumes. Using a corpus of 709k resumes from IT …
Investigating Cross-Linguistic Gender Bias in Hindi-English Across Domains
Measuring, evaluating and reducing Gender Bias has come to the forefront with newer and improved language embeddings being released every few months. But could this bias vary from domain to domain? We see a lot of work t…
TranslationGendered Language in Resumes and its Implications for Algorithmic Bias in Hiring
Despite growing concerns around gender bias in NLP models used in algorithmic hiring, there is little empirical work studying the extent and nature of gendered language in resumes.Using a corpus of 709k resumes from IT f…
FairnessMeasuring Gender Bias in Word Embeddings of Gendered Languages Requires Disentangling Grammatical Gender Signals
Does the grammatical gender of a language interfere when measuring the semantic gender information captured by its word embeddings? A number of anomalous gender bias measurements in the embeddings of gendered languages s…
Word EmbeddingsDegendering Resumes for Fair Algorithmic Resume Screening
We investigate whether it is feasible to remove gendered information from resumes to mitigate potential bias in algorithmic resume screening. Using a corpus of 709k resumes from IT firms, we first train a series of model…
Fairness