Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation
Gender bias is largely recognized as a problematic phenomenon affecting language technologies, with recent studies underscoring that it might surface differently across languages. However, most evaluation practices adopt a word-level focus on a narrow set of occupational nouns under synthetic conditions. Such protocols overlook key features of grammatical gender languages, which are characterized by morphosyntactic chains of gender agreement, marked on a variety of lexical items and parts-of-speech (POS). To overcome this limitation, we enrich the natural, gender-sensitive MuST-SHE corpus with two new annotation layers: POS and agreement chains. On this basis, we conduct multifaceted automatic and manual evaluations for three speech translation models, trained on varying amounts of data and different word segmentation techniques. Our work sheds light on model behaviours, gender bias, and its detection at several levels of granularity for English-French/Italian/Spanish.
Code (0)
등록된 구현이 없습니다.
Tasks
POSTranslationSimilar Papers 제목 키워드 기반
Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation
Gender bias is largely recognized as a problematic phenomenon affecting language technologies, with recent studies underscoring that it might surface differently across languages. However, most of current evaluation prac…
POSTranslationLLM Probe: Evaluating LLMs for Low-Resource Languages
Despite rapid advances in large language models (LLMs), their linguistic abilities in low-resource and morphologically rich languages are still not well understood due to limited annotated resources and the absence of st…
Speech RecognitionTo Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translation
We conduct a large-scale fine-grained comparative analysis of machine translations (MT) against human translations (HT) through the lens of morphosyntactic divergence. Across three language pairs and two types of diverge…
AttributeDiversityMachine TranslationTranslationPolish phonology and morphology through the lens of distributional semantics
This study investigates the relationship between the phonological and morphological structure of Polish words and their meanings using Distributional Semantics. In the present analysis, we ask whether there is a relation…
A theoretical model of dynamical grammatical gender shifting based on set-valued set function
This study investigates the diverse characteristics of nouns, focusing on both semantic (e.g., countable/uncountable) and morphosyntactic (e.g., masculine/feminine) distinctions. We explore inter-word variations for gend…