A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages
How should text dataset sizes be compared across languages? Even for content-matched (parallel) corpora, UTF-8 encoded text can require a dramatically different number of bytes for different languages. In our work, we define the byte premium between two languages as the ratio of bytes used to encode content-matched text in those languages. We compute byte premiums for 1155 languages, and we use linear regressions to estimate byte premiums for other languages. We release a tool to obtain byte premiums for any two languages, enabling comparisons of dataset sizes across languages for more equitable multilingual model development and data practices.
Code (1)
Similar Papers 제목 키워드 기반
Why do language models perform worse for morphologically complex languages?
Language models perform differently across languages. It has been previously suggested that morphological typology may explain some of this variability (Cotterell et al., 2018). We replicate previous analyses and find ad…
Language ModelingLanguage ModellingQuantifying Language Disparities in Multilingual Large Language Models
Results reported in large-scale multilingual evaluations are often fragmented and confounded by factors such as target languages, differences in experimental setups, and model choices. We propose a framework that disenta…
On the Impact of Random Seeds on the Fairness of Clinical Classifiers
Recent work has shown that fine-tuning large networks is surprisingly sensitive to changes in random seed(s). We explore the implications of this phenomenon for model fairness across demographic groups in clinical predic…
FairnessEveryone deserves their voice to be heard: Analyzing Predictive Gender Bias in ASR Models Applied to Dutch Speech Data
Recent research has shown that state-of-the-art (SotA) Automatic Speech Recognition (ASR) systems, such as Whisper, often exhibit predictive biases that disproportionately affect various demographic groups. This study fo…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)FairnessSemantic Similarity+3De-biasing "bias" measurement
When a model's performance differs across socially or culturally relevant groups--like race, gender, or the intersections of many such groups--it is often called "biased." While much of the work in algorithmic fairness o…
Decision MakingFairnessUncertainty Quantification