Interpreting Deep Learning Models with Marginal Attribution by Conditioning on Quantiles
A vastly growing literature on explaining deep learning models has emerged. This paper contributes to that literature by introducing a global gradient-based model-agnostic method, which we call Marginal Attribution by Conditioning on Quantiles (MACQ). Our approach is based on analyzing the marginal attribution of predictions (outputs) to individual features (inputs). Specificalllly, we consider variable importance by mixing (global) output levels and, thus, explain how features marginally contribute across different regions of the prediction space. Hence, MACQ can be seen as a marginal attribution counterpart to approaches such as accumulated local effects (ALE), which study the sensitivities of outputs by perturbing inputs. Furthermore, MACQ allows us to separate marginal attribution of individual features from interaction effect, and visually illustrate the 3-way relationship between marginal attribution, output level, and feature value.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Towards Faithful Neural Network Intrinsic Interpretation with Shapley Additive Self-Attribution
Self-interpreting neural networks have garnered significant interest in research. Existing works in this domain often (1) lack a solid theoretical foundation ensuring genuine interpretability or (2) compromise model expr…
Gaussian Copula Models for Nonignorable Missing Data Using Auxiliary Marginal Quantiles
We present an approach for modeling and imputation of nonignorable missing data. Our approach uses Bayesian data integration to combine (1) a Gaussian copula model for all study variables and missingness indicators, whic…
Data IntegrationImputationSelf-Attention Attribution: Interpreting Information Interactions Inside Transformer
The great success of Transformer-based models benefits from the powerful multi-head self-attention mechanism, which learns token dependencies and encodes contextual information from the input. Prior work strives to attri…
AttributeRobust Lambda-quantiles and extremal distributions
In this paper, we investigate the robust models for $\Lambda$-quantiles with partial information regarding the loss distribution, where $\Lambda$-quantiles extend the classical quantiles by replacing the fixed probabilit…
Initialization Noise in Image Gradients and Saliency Maps
In this paper, we examine gradients of logits of image classification CNNs by input pixel values. We observe that these fluctuate considerably with training randomness, such as the random initialization of the networ…
image-classificationImage Classification