Revealing Similar Semantics Inside CNNs: An Interpretable Concept-based Comparison of Feature Spaces
Safety-critical applications require transparency in artificial intelligence (AI) components, but widely used convolutional neural networks (CNNs) widely used for perception tasks lack inherent interpretability. Hence, insights into what CNNs have learned are primarily based on performance metrics, because these allow, e.g., for cross-architecture CNN comparison. However, these neglect how knowledge is stored inside. To tackle this yet unsolved problem, our work proposes two methods for estimating the layer-wise similarity between semantic information inside CNN latent spaces. These allow insights into both the flow and likeness of semantic information within CNN layers, and into the degree of their similarity between different network architectures. As a basis, we use two renowned explainable artificial intelligence (XAI) techniques, which are used to obtain concept activation vectors, i.e., global vector representations in the latent space. These are compared with respect to their activation on test inputs. When applied to three diverse object detectors and two datasets, our methods reveal that (1) similar semantic concepts are learned regardless of the CNN architecture, and (2) similar concepts emerge in similar relative layer depth, independent of the total number of layers. Finally, our approach poses a promising step towards semantic model comparability and comprehension of how different CNNs process semantic information.
Code (0)
등록된 구현이 없습니다.
Tasks
Explainable artificial intelligenceExplainable Artificial Intelligence (XAI)Model SelectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Interpretable Convolutional Neural Networks
This paper proposes a method to modify traditional convolutional neural networks (CNNs) into interpretable CNNs, in order to clarify knowledge representations in high conv-layers of CNNs. In an interpretable CNN, each fi…
Objectsingle catogory classificationAn Interpretable Neuron Embedding for Static Knowledge Distillation
Although deep neural networks have shown well-performance in various tasks, the poor interpretability of the models is always criticized. In the paper, we propose a new interpretable neural network method, by embedding n…
Knowledge DistillationMaking CNNs Interpretable by Building Dynamic Sequential Decision Forests with Top-down Hierarchy Learning
In this paper, we propose a generic model transfer scheme to make Convlutional Neural Networks (CNNs) interpretable, while maintaining their high classification accuracy. We achieve this by building a differentiable deci…
ClassificationNetwork Inversion of Convolutional Neural Nets
Neural networks have emerged as powerful tools across various applications, yet their decision-making process often remains opaque, leading to them being perceived as "black boxes." This opacity raises concerns about the…
Decision MakingDiversityVisualizing Random Forest with Self-Organising Map
Random Forest (RF) is a powerful ensemble method for classification and regression tasks. It consists of decision trees set. Although, a single tree is well interpretable for human, the ensemble of trees is a black-box m…
ClassificationGeneral Classification