Evaluating Local Explanations using White-box Models
Evaluating explanation techniques using human subjects is costly, time-consuming and can lead to subjectivity in the assessments. To evaluate the accuracy of local explanations, we require access to the true feature importance scores for a given instance. However, the prediction function of a model usually does not decompose into linear additive terms that indicate how much a feature contributes to the output. In this work, we suggest to instead focus on the log odds ratio (LOR) of the prediction function, which naturally decomposes into additive terms for logistic regression and naive Bayes. We demonstrate how we can benchmark different explanation techniques in terms of their similarity to the LOR scores based on our proposed approach. In the experiments, we compare prominent local explanation techniques and find that the performance of the techniques can depend on the underlying model, the dataset, which data point is explained, the normalization of the data and the similarity metric.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature ImportanceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Evaluating Attribution Methods using White-Box LSTMs
Interpretability methods for neural networks are difficult to evaluate because we do not understand the black-box models typically used to test them. This paper proposes a framework in which interpretability methods are …
SLISEMAP: Supervised dimensionality reduction through local explanations
Existing methods for explaining black box learning models often focus on building local explanations of model behaviour for a particular data item. It is possible to create global explanations for all data items, but the…
ClassificationDimensionality ReductionExplainable ModelsGPU+1Multi-Domain Explainability of Preferences
Preference mechanisms, such as human preference, LLM-as-a-Judge (LaaJ), and reward models, are central to aligning and evaluating large language models (LLMs). Yet, the underlying concepts that drive these preferences re…
The Blame Problem in Evaluating Local Explanations, and How to Tackle it
The number of local model-agnostic explanation techniques proposed has grown rapidly recently. One main reason is that the bar for developing new explainability techniques is low due to the lack of optimal evaluation mea…
Pathologist-Like Explanations Unveiled: an Explainable Deep Learning System for White Blood Cell Classification
White blood cells (WBCs) play a crucial role in safeguarding the human body against pathogens and foreign substances. Leveraging the abundance of WBC imaging data and the power of deep learning algorithms, automated WBC …
Classification