Toward Deconfounding the Influence of Entity Demographics for Question Answering Accuracy
The goal of question answering (QA) is to answer any question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, model accuracy analysis reveals little evidence that accuracy is lower for people based on gender or nationality; instead, there is more variation on professions (question topic). But QA's lack of representation could itself hide evidence of bias, necessitating QA datasets that better represent global diversity.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityQuestion AnsweringSimilar Papers 제목 키워드 기반
Toward Deconfounding the Effect of Entity Demographics for Question Answering Accuracy
The goal of question answering (QA) is to answer _any_ question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, an analysis of model accuracy reveals lit…
DiversityQuestion AnsweringWhen Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset
Annotators are not fungible. Their demographics, life experiences, and backgrounds all contribute to how they label data. However, NLP has only recently considered how annotator identity might influence their decisions. …
Question AnsweringJointly De-biasing Face Recognition and Demographic Attribute Estimation
We address the problem of bias in automated face recognition and demographic attribute estimation algorithms, where errors are lower on certain cohorts belonging to specific demographic groups. We present a novel de-bias…
AttributeFace RecognitionWhich Demographics do LLMs Default to During Annotation?
Demographics and cultural background of annotators influence the labels they assign in text annotation -- for instance, an elderly woman might find it offensive to read a message addressed to a "bro", but a male teenager…
text annotationMixing Context Granularities for Improved Entity Linking on Question Answering Data across Entity Categories
The first stage of every knowledge base question answering approach is to link entities in the input question. We investigate entity linking in the context of a question answering task and present a jointly optimized neu…
Entity DisambiguationEntity LinkingKnowledge Base Question AnsweringQuestion Answering