Measuring algorithmic interpretability: A human-learning-based framework and the corresponding cognitive complexity score
Algorithmic interpretability is necessary to build trust, ensure fairness, and track accountability. However, there is no existing formal measurement method for algorithmic interpretability. In this work, we build upon programming language theory and cognitive load theory to develop a framework for measuring algorithmic interpretability. The proposed measurement framework reflects the process of a human learning an algorithm. We show that the measurement framework and the resulting cognitive complexity score have the following desirable properties - universality, computability, uniqueness, and monotonicity. We illustrate the measurement framework through a toy example, describe the framework and its conceptual underpinnings, and demonstrate the benefits of the framework, in particular for managers considering tradeoffs when selecting algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
FairnessSimilar Papers 제목 키워드 기반
Algorithmic Insurance
As machine learning algorithms start to get integrated into the decision-making process of companies and organizations, insurance products are being developed to protect their owners from liability risk. Algorithmic liab…
Binary ClassificationBreast Cancer DetectionDecision MakingRepresentation Fidelity:Auditing Algorithmic Decisions About Humans Using Self-Descriptions
This paper introduces a new dimension for validating algorithmic decisions about humans by measuring the fidelity of their representations. Representation Fidelity measures if decisions about a person rest on reasonable …
The FairCeptron: A Framework for Measuring Human Perceptions of Algorithmic Fairness
Measures of algorithmic fairness often do not account for human perceptions of fairness that can substantially vary between different sociodemographics and stakeholders. The FairCeptron framework is an approach for study…
Decision MakingFairnessTowards causal benchmarking of bias in face analysis algorithms
Measuring algorithmic bias is crucial both to assess algorithmic fairness, and to guide the improvement of algorithms. Current methods to measure algorithmic bias in computer vision, which are based on observational data…
AttributeBenchmarkingFairnessGender ClassificationGCI: A (G)raph (C)oncept (I)nterpretation Framework
Explainable AI (XAI) underwent a recent surge in research on concept extraction, focusing on extracting human-interpretable concepts from Deep Neural Networks. An important challenge facing concept extraction approaches …
Explainable Artificial Intelligence (XAI)Molecular Property PredictionPredictionProperty Prediction