Whose wife is it anyway? Assessing bias against same-gender relationships in machine translation
Machine translation often suffers from biased data and algorithms that can lead to unacceptable errors in system output. While bias in gender norms has been investigated, less is known about whether MT systems encode bias about social relationships, e.g., "the lawyer kissed her wife." We investigate the degree of bias against same-gender relationships in MT systems, using generated template sentences drawn from several noun-gender languages (e.g., Spanish) and comprised of popular occupation nouns. We find that three popular MT services consistently fail to accurately translate sentences concerning relationships between entities of the same gender. The error rate varies considerably based on the context, and same-gender sentences referencing high female-representation occupations are translated with lower accuracy. We provide this work as a case study in the evaluation of intrinsic bias in NLP systems with respect to social relationships.
Code (1)
Tasks
Machine TranslationSimilar Papers 제목 키워드 기반
Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
Federated Learning (FL) allows multiple participants to train machine learning models collaboratively by keeping their datasets local while only exchanging model updates. Alas, this is not necessarily free from privacy a…
Federated LearningUsing matrices to model symbolic relationship
We describe a way of learning matrix representations of objects and relationships. The goal of learning is to allow multiplication of matrices to represent symbolic relationships between objects and symbolic relationship…
model``Let Everything Turn Well in Your Wife'': Generation of Adult Humor Using Lexical Constraints
Midwifery Learning and Forecasting: Predicting Content Demand with User-Generated Logs
Every day, 800 women and 6,700 newborns die from complications related to pregnancy or childbirth. A well-trained midwife can prevent most of these maternal and newborn deaths. Data science models together with logs gene…
What is a protest anyway? Codebook conceptualization is still a first-order concern in LLM-era classification
Generative large language models (LLMs) are now used extensively for text classification in computational social science (CSS). In this work, focus on the steps before and after LLM prompting -- conceptualization of conc…
Text Classification