GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered Languages
Neural Machine Translation (NMT) continues to improve in quality and adoption, yet the inadvertent perpetuation of gender bias remains a significant concern. Despite numerous studies on gender bias in translations into English from weakly gendered-languages, there are no benchmarks for evaluating this phenomenon or for assessing mitigation strategies. To address this gap, we introduce GATE X-E, an extension to the GATE (Rarrick et al., 2023) corpus, that consists of human translations from Turkish, Hungarian, Finnish, and Persian into English. Each translation is accompanied by feminine, masculine, and neutral variants. The dataset, which contains between 1250 and 1850 instances for each of the four language pairs, features natural sentences with a wide range of sentence lengths and domains, challenging translation rewriters on various linguistic phenomena. Additionally, we present a translation gender rewriting solution built with GPT-4 and use GATE X-E to evaluate it. We open source our contributions to encourage further research on gender debiasing.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTSentenceTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FairQE: Multi-Agent Framework for Mitigating Gender Bias in Translation Quality Estimation
Quality Estimation (QE) aims to assess machine translation quality without reference translations, but recent studies have shown that existing QE models exhibit systematic gender bias. In particular, they tend to favor m…
Machine TranslationA Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation
Recent instruction fine-tuned models can solve multiple NLP tasks when prompted to do so, with machine translation (MT) being a prominent use case. However, current research often focuses on standard performance benchmar…
FairnessFew-Shot LearningMachine TranslationTranslationSignificance of Chain of Thought in Gender Bias Mitigation for English-Dravidian Machine Translation
Gender bias in machine translation (MT) sys- tems poses a significant challenge to achieving accurate and inclusive translations. This paper examines gender bias in machine translation systems for languages such as Telug…
FairnessMachine TranslationTranslationGender-specific Machine Translation with Large Language Models
While machine translation (MT) systems have seen significant improvements, it is still common for translations to reflect societal biases, such as gender bias. Decoder-only Large Language Models (LLMs) have demonstrated …
coreference-resolutionCoreference ResolutionDecoderIn-Context Learning+3GATE: A Challenge Set for Gender-Ambiguous Translation Examples
Although recent years have brought significant progress in improving translation of unambiguously gendered sentences, translation of ambiguously gendered input remains relatively unexplored. When source gender is ambiguo…
Machine TranslationTranslation