Ask-Before-Detection: Identifying and Mitigating Conformity Bias in LLM-Powered Error Detector for Math Word Problem Solutions
The rise of large language models (LLMs) offers new opportunities for automatic error detection in education, particularly for math word problems (MWPs). While prior studies demonstrate the promise of LLMs as error detectors, they overlook the presence of multiple valid solutions for a single MWP. Our preliminary analysis reveals a significant performance gap between conventional and alternative solutions in MWPs, a phenomenon we term conformity bias in this work. To mitigate this bias, we introduce the Ask-Before-Detect (AskBD) framework, which generates adaptive reference solutions using LLMs to enhance error detection. Experiments on 200 examples of GSM8K show that AskBD effectively mitigates bias and improves performance, especially when combined with reasoning-enhancing techniques like chain-of-thought prompting.
Code (0)
등록된 구현이 없습니다.
Tasks
GSM8KMathvalidSimilar Papers 제목 키워드 기반
Debiased Contrastive Representation Learning for Mitigating Dual Biases in Recommender Systems
In recommender systems, popularity and conformity biases undermine recommender effectiveness by disproportionately favouring popular items, leading to their over-representation in recommendation lists and causing an unba…
Contrastive LearningDiversityRecommendation SystemsRepresentation LearningTowards Trustworthy Audio Deepfake Detection: A Systematic Framework for Diagnosing and Mitigating Gender Bias
Audio deepfake detection systems are increasingly deployed in high-stakes security applications, yet their fairness across demographic groups remains critically underexamined. Prior work measures gender disparity but doe…
Audio Deepfake DetectionPassivity-based Analysis and Design for Population Dynamics with Conformity Biases
This paper addresses mechanisms for boundedly rational decision makers in discrete choice problem. First, we introduce two mathematical models of population dynamics with conformity biases. We next analyze the models in …
Detecting Melanoma Fairly: Skin Tone Detection and Debiasing for Skin Lesion Classification
Convolutional Neural Networks have demonstrated human-level performance in the classification of melanoma and other skin lesions, but evident performance disparities between differing skin tones should be addressed befor…
Lesion ClassificationSkin Lesion ClassificationUPV at CheckThat! 2021: Mitigating Cultural Differences for Identifying Multilingual Check-worthy Claims
Identifying check-worthy claims is often the first step of automated fact-checking systems. Tackling this task in a multilingual setting has been understudied. Encoding inputs with multilingual text representations could…
Fact CheckingLanguage Identification