MRCLens: an MRC Dataset Bias Detection Toolkit
Many recent neural models have shown remarkable empirical results in Machine Reading Comprehension, but evidence suggests sometimes the models take advantage of dataset biases to predict and fail to generalize on out-of-sample data. While many other approaches have been proposed to address this issue from the computation perspective such as new architectures or training procedures, we believe a method that allows researchers to discover biases, and adjust the data or the models in an earlier stage will be beneficial. Thus, we introduce MRCLens, a toolkit that detects whether biases exist before users train the full model. For the convenience of introducing the toolkit, we also provide a categorization of common biases in MRC.
Code (0)
등록된 구현이 없습니다.
Tasks
Bias DetectionMachine Reading ComprehensionReading ComprehensionSimilar Papers 제목 키워드 기반
MRCLens: an MRC Dataset Bias Detection Toolkit
Many recent neural models have shown remarkable empirical results in Machine Reading Comprehension, but evidence suggests sometimes the models take advantage of dataset biases to predict and fail to generalize on out-of-…
Bias DetectionMachine Reading ComprehensionReading ComprehensionUnBias-Plus: Detect, Explain, and Rewrite Bias
Bias in natural language remains a persistent challenge in both human-written and AI-generated content, affecting domains such as journalism, education, and AI research. Most existing detection methods identify only the …
humancompatible.detect: a Python Toolkit for Detecting Bias in AI Models
There is a strong recent emphasis on trustworthy AI. In particular, international regulations, such as the AI Act, demand that AI practitioners measure data quality on the input and estimate bias on the output of high-ri…
Bias DetectionMT-LENS: An all-in-one Toolkit for Better Machine Translation Evaluation
We introduce MT-LENS, a framework designed to evaluate Machine Translation (MT) systems across a variety of tasks, including translation quality, gender bias detection, added toxicity, and robustness to misspellings. Whi…
AllBenchmarkingBias DetectionGender Bias Detection+3From Expectation to Habit: Why Do Software Practitioners Adopt Fairness Toolkits?
As the adoption of machine learning (ML) systems continues to grow across industries, concerns about fairness and bias in these systems have taken center stage. Fairness toolkits, designed to mitigate bias in ML models, …
Fairness