paper-with-me

Papers

Investigating Gender Bias in Turkish Language Models

2024-04-17 · Orhun Caglidil, Malte Ostendorff, Georg Rehm

Language models are trained mostly on Web data, which often contains social stereotypes and biases that the models can inherit. This has potentially negative consequences, as models can amplify these biases in downstream tasks or applications. However, prior research has primarily focused on the English language, especially in the context of gender bias. In particular, grammatically gender-neutral languages such as Turkish are underexplored despite representing different linguistic properties to language models with possibly different effects on biases. In this paper, we fill this research gap and investigate the significance of gender bias in Turkish language models. We build upon existing bias evaluation frameworks and extend them to the Turkish language by translating existing English tests and creating new ones designed to measure gender bias in the context of T\"urkiye. Specifically, we also evaluate Turkish language models for their embedded ethnic bias toward Kurdish people. Based on the experimental results, we attribute possible biases to different model characteristics such as the model size, their multilingualism, and the training corpora. We make the Turkish gender bias dataset publicly available.

📄 PDF Abstract BibTeX arXiv:2404.11726

Code (1)

malteos/turkish-lm-bias 공식 구현 pytorch

Tasks

Attribute

Similar Papers 제목 키워드 기반

Investigating Markers and Drivers of Gender Bias in Machine Translations

2024-03-18 · Peter J Barclay, Ashkan Sami

Implicit gender bias in Large Language Models (LLMs) is a well-documented problem, and implications of gender introduced into automatic translations can perpetuate real-world biases. However, some LLMs use heuristics or …

SentenceTranslation

Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Models

2021-08-23 · INLG (ACL) 2021 8 · Chloe Ciora, Nur Iren, Malihe Alikhani

As Machine Translation (MT) has become increasingly more powerful, accessible, and widespread, the potential for the perpetuation of bias has grown alongside its advances. While overt indicators of bias have been studied…

Machine TranslationTranslation

Evaluating Gender Bias in the Translation of Gender-Neutral Languages into English

2023-11-15 · Spencer Rarrick, Ranjita Naik, Sundar Poudel, Vishal Chowdhary

Machine Translation (MT) continues to improve in quality and adoption, yet the inadvertent perpetuation of gender bias remains a significant concern. Despite numerous studies into gender bias in translations from gender-…

Machine TranslationSentenceTranslation

GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered Languages

2024-02-22 · Spencer Rarrick, Ranjita Naik, Sundar Poudel, Vishal Chowdhary

Neural Machine Translation (NMT) continues to improve in quality and adoption, yet the inadvertent perpetuation of gender bias remains a significant concern. Despite numerous studies on gender bias in translations into E…

Machine TranslationNMTSentenceTranslation

Performance of Objective Speech Quality Metrics on Languages Beyond Validation Data: A Study of Turkish and Korean

2025-05-22 · Javier Perez, Dimme de Groot, Jorge Martinez

Objective speech quality measures are widely used to assess the performance of video conferencing platforms and telecommunication systems. They predict human-rated speech quality and are crucial for assessing the systems…