On Sparse Critical Paths of Neural Response
Is critical input information encoded in specific sparse paths within the network? The pruning objective --- finding a subset of neurons for which the response remains unchanged --- has been used to discover such paths. However, we show that paths obtained from this objective do not necessarily encode the input features and also encompass (dead) neurons that were not originally contributing to the response. We investigate selecting paths based on neurons' contributions to the response to ensure that the paths envelop the critical segments of the encoded input information. We show that these paths have the property of being provably locally linear in an $\ell_{2}$-ball of the input, thus having stable gradients. This property is leveraged for proposing a feature attribution paradigm that is guided by neurons, therefore inherently taking interactions between input features into account. We evaluate the attribution methodology quantitatively in mainstream benchmarks.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Pulse Shape-Aided Multipath Delay Estimation for Fine-Grained WiFi Sensing
Due to the finite bandwidth of practical wireless systems, one multipath component can manifest itself as a discrete pulse consisting of multiple taps in the digital delay domain. This effect is called channel leakage, w…
BenchmarkingInterpret Neural Networks by Identifying Critical Data Routing Paths
Interpretability of a deep neural network aims to explain the rationale behind its decisions and enable the users to understand the intelligent agents, which has become an important issue due to its importance in practic…
Neural Response Interpretation through the Lens of Critical Pathways
Is critical input information encoded in specific sparse pathways within the neural network? In this work, we discuss the problem of identifying these critical pathways and subsequently leverage them for interpreting the…
Identification of RIS-Assisted Paths for Wireless Integrated Sensing and Communication
Distinguishing between reconfigurable intelligent surface (RIS) assisted paths and non-line-of-sight (NLOS) paths is a fundamental problem for RIS-assisted integrated sensing and communication. In this work, we propose a…
Integrated sensing and communicationStruProKGR: A Structural and Probabilistic Framework for Sparse Knowledge Graph Reasoning
Sparse Knowledge Graphs (KGs) are commonly encountered in real-world applications, where knowledge is often incomplete or limited. Sparse KG reasoning, the task of inferring missing knowledge over sparse KGs, is inherent…
Knowledge Graphs