Global Performance Disparities Between English-Language Accents in Automatic Speech Recognition
Past research has identified discriminatory automatic speech recognition (ASR) performance as a function of the racial group and nationality of the speaker. In this paper, we expand the discussion beyond bias as a function of the individual national origin of the speaker to look for bias as a function of the geopolitical orientation of their nation of origin. We audit some of the most popular English language ASR services using a large and global data set of speech from The Speech Accent Archive, which includes over 2,700 speakers of English born in 171 different countries. We show that, even when controlling for multiple linguistic covariates, ASR service performance has a statistically significant relationship to the political alignment of the speaker's birth country with respect to the United States' geopolitical power. This holds for all ASR services tested. We discuss this bias in the context of the historical use of language to maintain global and political power.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unintended Impacts of LLM Alignment on Global Representation
Before being deployed for user-facing applications, developers align Large Language Models (LLMs) to user preferences through a variety of procedures, such as Reinforcement Learning From Human Feedback (RLHF) and Direct …
Instruction FollowingThe World According to LLMs: How Geographic Origin Influences LLMs' Entity Deduction Capabilities
Large Language Models (LLMs) have been extensively tuned to mitigate explicit biases, yet they often exhibit subtle implicit biases rooted in their pre-training data. Rather than directly probing LLMs with human-crafted …
A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems
Achieving robust language technologies that can perform well across the world's many languages is a central goal of multilingual NLP. In this work, we take stock of and empirically analyse task performance disparities th…
Language ModelingLanguage ModellingMultilingual NLPTask-Oriented Dialogue SystemsThink Globally, Group Locally: Evaluating LLMs Using Multi-Lingual Word Grouping Games
Large language models (LLMs) can exhibit biases in reasoning capabilities due to linguistic modality, performing better on tasks in one language versus another, even with similar content. Most previous works evaluate thi…
Artificially Fluent: Swahili AI Performance Benchmarks Between English-Trained and Natively-Trained Datasets
As large language models (LLMs) expand multilingual capabilities, questions remain about the equity of their performance across languages. While many communities stand to benefit from AI systems, the dominance of English…