NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks
The problem of explaining deep learning models, and model predictions generally, has attracted intensive interest recently. Many successful approaches forgo global approximations in order to provide more faithful local interpretations of the model's behavior. LIME develops multiple interpretable models, each approximating a large neural network on a small region of the data manifold and SP-LIME aggregates the local models to form a global interpretation. Extending this line of research, we propose a simple yet effective method, NormLIME for aggregating local models into global and class-specific interpretations. A human user study strongly favored class-specific interpretations created by NormLIME to other feature importance metrics. Numerical experiments confirm that NormLIME is effective at recognizing important features.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature ImportanceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Inferring feature importance with uncertainties in high-dimensional data
Estimating feature importance is a significant aspect of explaining data-based models. Besides explaining the model itself, an equally relevant question is which features are important in the underlying data generating p…
Feature ImportanceVocal Bursts Intensity PredictionNonparametric variable importance using an augmented neural network with multi-task learning
In predictive modeling applications, it is often of interest to determine the relative contribution of subsets of features in explaining the variability of an outcome. It is useful to consider this variable importan…
Multi-Task LearningExplaining COVID-19 and Thoracic Pathology Model Predictions by Identifying Informative Input Features
Neural networks have demonstrated remarkable performance in classification and regression tasks on chest X-rays. In order to establish trust in the clinical routine, the networks' prediction mechanism needs to be interpr…
Feature ImportancePredictionregressionSum-of-Parts: Faithful Attributions for Groups of Features
Feature attributions explain machine learning predictions by assigning importance scores to input features. While faithful attributions accurately reflect feature contributions to the model's prediction, unfaithful ones …
Decision Makingscientific discoveryCan local explanation techniques explain linear additive models?
Local model-agnostic additive explanation techniques decompose the predicted output of a black-box model into additive feature importance scores. Questions have been raised about the accuracy of the produced local additi…
Additive modelsExplanation Fidelity EvaluationFeature Importance