Toward Deconfounding the Effect of Entity Demographics for Question Answering Accuracy
The goal of question answering (QA) is to answer _any_ question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, an analysis of model accuracy reveals little evidence that accuracy is lower for people based on gender or nationality; instead, there is more variation on professions (question topic) and question ambiguity. But QA’s lack of representation could itself hide evidence of bias, necessitating QA datasets that better represent global diversity.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityQuestion AnsweringSimilar Papers 제목 키워드 기반
Toward Deconfounding the Influence of Entity Demographics for Question Answering Accuracy
The goal of question answering (QA) is to answer any question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, model accuracy analysis reveals little evid…
DiversityQuestion AnsweringWhen Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset
Annotators are not fungible. Their demographics, life experiences, and backgrounds all contribute to how they label data. However, NLP has only recently considered how annotator identity might influence their decisions. …
Question AnsweringEmerging Challenges in Personalized Medicine: Assessing Demographic Effects on Biomedical Question Answering Systems
State-of-the-art question answering (QA) models exhibit a variety of social biases (e.g., with respect to sex or race), generally explained by similar issues in their training data. However, what has been overlooked so f…
FairnessQuestion AnsweringSimple Question Answering by Attentive Convolutional Neural Network
This work focuses on answering single-relation factoid questions over Freebase. Each question can acquire the answer from a single fact of form (subject, predicate, object) in Freebase. This task, simple question answeri…
Entity LinkingFact SelectionQuestion AnsweringMixing Context Granularities for Improved Entity Linking on Question Answering Data across Entity Categories
The first stage of every knowledge base question answering approach is to link entities in the input question. We investigate entity linking in the context of a question answering task and present a jointly optimized neu…
Entity DisambiguationEntity LinkingKnowledge Base Question AnsweringQuestion Answering