Improving Feature Attribution through Input-specific Network Pruning
Attributing the output of a neural network to the contribution of given input elements is a way of shedding light on the black-box nature of neural networks. Due to the complexity of current network architectures, current gradient-based attribution methods provide very noisy or coarse results. We propose to prune a neural network for a given single input to keep only neurons that highly contribute to the prediction. We show that by input-specific pruning, network gradients change from reflecting local (noisy) importance information to global importance. Our proposed method is efficient and generates fine-grained attribution maps. We further provide a theoretical justification of the pruning approach relating it to perturbations and validate it through a novel experimental setup. Our method is evaluated by multiple benchmarks: sanity checks, pixel perturbation, and Remove-and-Retrain (ROAR). These benchmarks evaluate the method from different perspectives and our method performs better than other methods across all evaluations.
Code (0)
등록된 구현이 없습니다.
Tasks
Network PruningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On Sparse Critical Paths of Neural Response
Is critical input information encoded in specific sparse paths within the network? The pruning objective --- finding a subset of neurons for which the response remains unchanged --- has been used to discover such paths. …
Neural Response Interpretation through the Lens of Critical Pathways
Is critical input information encoded in specific sparse pathways within the neural network? In this work, we discuss the problem of identifying these critical pathways and subsequently leverage them for interpreting the…
SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance
The leap in performance in state-of-the-art computer vision methods is attributed to the development of deep neural networks. However it often comes at a computational price which may hinder their deployment. To alleviat…
CachePrune: Neural-Based Attribution Defense Against Indirect Prompt Injection Attacks
Large Language Models (LLMs) are identified as being susceptible to indirect prompt injection attack, where the model undesirably deviates from user-provided instructions by executing tasks injected in the prompt context…
Instruction FollowingTimeSHAP: Explaining Recurrent Models through Sequence Perturbations
Although recurrent neural networks (RNNs) are state-of-the-art in numerous sequential decision-making tasks, there has been little research on explaining their predictions. In this work, we present TimeSHAP, a model-agno…
Decision MakingFeature ImportanceFraud DetectionSequential Decision Making+2