MORPHOGEN: A Multilingual Benchmark for Evaluating Gender-Aware Morphological Generation

2026-04-20 · Mehul Agarwal, Aditya Aggarwal, Arnav Goel, Medha Hira, Anubha Gupta arxiv

While multilingual large language models (LLMs) perform well on high-level tasks like translation and question answering, their ability to handle grammatical gender and morphological agreement remains underexplored. In morphologically rich languages, gender influences verb conjugation, pronouns, and even first-person constructions with explicit and implicit mentions of gender. We introduce MORPHOGEN, a morphologically grounded large-scale benchmark dataset for evaluating gender-aware generation in three typologically diverse grammatically gendered languages: French, Arabic, and Hindi. The core task, GENFORM, requires models to rewrite a first-person sentence in the opposite gender while preserving its meaning and structure. We construct a high-quality synthetic dataset spanning these three languages and benchmark 15 popular multilingual LLMs (2B-70B) on their ability to perform this transformation. Our results reveal significant gaps and interesting insights into how current models handle morphological gender. MORPHOGEN provides a focused diagnostic lens for gender-aware language modeling and lays the groundwork for future research on inclusive and morphology-sensitive NLP.

📄 PDF Abstract BibTeX arXiv:2604.18914

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering

On Evaluating and Mitigating Gender Biases in Multilingual Settings

2023-07-04 · Aniket Vashishtha, Kabir Ahuja, Sunayana Sitaram

While understanding and removing gender biases in language models has been a long-standing problem in Natural Language Processing, prior research work has primarily been limited to English. In this work, we investigate s…

ConGA: Guidelines for Contextual Gender Annotation. A Framework for Annotating Gender in Machine Translation

2026-03-18 · Argentina Anna Rescigno, Eva Vanmassenhove, Johanna Monti arxiv

Handling gender across languages remains a persistent challenge for Machine Translation (MT) and Large Language Models (LLMs), especially when translating from gender-neutral languages into morphologically gendered ones,…

Machine Translation

Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translation

2023-05-23 · Minwoo Lee, Hyukhun Koh, Kang-il Lee, Dongdong Zhang 외

Gender bias is a significant issue in machine translation, leading to ongoing research efforts in developing bias mitigation techniques. However, most works focus on debiasing bilingual models without much consideration …

Contrastive Learning Machine Translation Translation

IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia

2026-03-18 · Priyaranjan Pattnayak, Sanchari Chowdhuri arxiv

As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages remains poorly understood. We present the first systematic evaluation of LLM safe…

Multilingual Holistic Bias: Extending Descriptors and Patterns to Unveil Demographic Biases in Languages at Scale

2023-05-22 · Marta R. Costa-jussà, Pierre Andrews, Eric Smith, Prangthip Hansanti 외

We introduce a multilingual extension of the HOLISTICBIAS dataset, the largest English template-based taxonomy of textual people references: MULTILINGUALHOLISTICBIAS. This extension consists of 20,459 sentences in 50 lan…

Joint Multilingual Sentence Representations Sentence