Can Question Generation Debias Question Answering Models? A Case Study on Question-Context Lexical Overlap
Question answering (QA) models for reading comprehension have been demonstrated to exploit unintended dataset biases such as question-context lexical overlap. This hinders QA models from generalizing to under-represented samples such as questions with low lexical overlap. Question generation (QG), a method for augmenting QA datasets, can be a solution for such performance degradation if QG can properly debias QA datasets. However, we discover that recent neural QG models are biased towards generating questions with high lexical overlap, which can amplify the dataset bias. Moreover, our analysis reveals that data augmentation with these QG models frequently impairs the performance on questions with low lexical overlap, while improving that on questions with high lexical overlap. To address this problem, we use a synonym replacement-based approach to augment questions with low lexical overlap. We demonstrate that the proposed data augmentation approach is simple yet effective to mitigate the degradation problem with only 70k synthetic examples. Our data is publicly available at https://github.com/KazutoshiShinoda/Synonym-Replacement.
Code (1)
Tasks
Data AugmentationQuestion AnsweringQuestion GenerationQuestion-GenerationReading ComprehensionSimilar Papers 제목 키워드 기반
Can Question Generation Debias Question Answering Models? A Case Study on Question–Context Lexical Overlap
Question answering (QA) models for reading comprehension have been demonstrated to exploit unintended dataset biases such as question–context lexical overlap. This hinders QA models from generalizing to under-represented…
Data AugmentationQuestion AnsweringQuestion GenerationQuestion-Generation+1DeCAP: Context-Adaptive Prompt Generation for Debiasing Zero-shot Question Answering in Large Language Models
While Large Language Models (LLMs) excel in zero-shot Question Answering (QA), they tend to expose biases in their internal knowledge when faced with socially sensitive questions, leading to a degradation in performance.…
FairnessQuestion AnsweringAdaptive loose optimization for robust question answering
Question answering methods are well-known for leveraging data bias, such as the language prior in visual question answering and the position bias in machine reading comprehension (extractive question answering). Current …
Extractive Question-AnsweringMachine Reading ComprehensionQuestion AnsweringReading Comprehension+2QIRL: Boosting Visual Question Answering via Optimized Question-Image Relation Learning
Existing debiasing approaches in Visual Question Answering (VQA) primarily focus on enhancing visual learning, integrating auxiliary models, or employing data augmentation strategies. However, these methods exhibit two m…
Data AugmentationImage GenerationQuestion AnsweringRelation+3Pseudo Ambiguous and Clarifying Questions Based on Sentence Structures Toward Clarifying Question Answering System
Question answering (QA) with disambiguation questions is essential for practical QA systems because user questions often do not contain information enough to find their answers. We call this task clarifying question answ…
Question AnsweringQuestion GenerationQuestion-GenerationSentence