A Seven-Layer Model for Standardising AI Fairness Assessment
Problem statement: Standardisation of AI fairness rules and benchmarks is challenging because AI fairness and other ethical requirements depend on multiple factors such as context, use case, type of the AI system, and so on. In this paper, we elaborate that the AI system is prone to biases at every stage of its lifecycle, from inception to its usage, and that all stages require due attention for mitigating AI bias. We need a standardised approach to handle AI fairness at every stage. Gap analysis: While AI fairness is a hot research topic, a holistic strategy for AI fairness is generally missing. Most researchers focus only on a few facets of AI model-building. Peer review shows excessive focus on biases in the datasets, fairness metrics, and algorithmic bias. In the process, other aspects affecting AI fairness get ignored. The solution proposed: We propose a comprehensive approach in the form of a novel seven-layer model, inspired by the Open System Interconnection (OSI) model, to standardise AI fairness handling. Despite the differences in the various aspects, most AI systems have similar model-building stages. The proposed model splits the AI system lifecycle into seven abstraction layers, each corresponding to a well-defined AI model-building or usage stage. We also provide checklists for each layer and deliberate on potential sources of bias in each layer and their mitigation methodologies. This work will facilitate layer-wise standardisation of AI fairness rules and benchmarking parameters.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingFairnessSimilar Papers 제목 키워드 기반
A Comprehensive Survey and Classification of Evaluation Criteria for Trustworthy Artificial Intelligence
This paper presents a systematic review of the literature on evaluation criteria for Trustworthy Artificial Intelligence (TAI), with a focus on the seven EU principles of TAI. This systematic literature review identifies…
Systematic Literature ReviewExplanation Multiplicity: Circuit-Level Interpretability Evidence Does Not Survive Defensible Analytic Variation
The EU AI Act requires providers of high-risk systems to file technical documentation describing how the system reaches its decisions. Mechanistic interpretability is the obvious source of such evidence, and circuit disc…
Fair Representation: Guaranteeing Approximate Multiple Group Fairness for Unknown Tasks
Motivated by scenarios where data is used for diverse prediction tasks, we study whether fair representation can be used to guarantee fairness for unknown tasks and for multiple fairness notions simultaneously. We consid…
AllFairnessvalidUniversalCEFR: Enabling Open Multilingual Research on Language Proficiency Assessment
We introduce UniversalCEFR, a large-scale multilingual multidimensional dataset of texts annotated according to the CEFR (Common European Framework of Reference) scale in 13 languages. To enable open research in both aut…
Fair Enough: Searching for Sufficient Measures of Fairness
Testing machine learning software for ethical bias has become a pressing current concern. In response, recent research has proposed a plethora of new fairness metrics, for example, the dozens of fairness metrics in the I…
Fairness