VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision Graph Neural Network

2022-09-15 · Zhenpeng Feng, Xiyang Cui, Hongbing Ji, Mingzhe Zhu, Ljubisa Stankovic

Graph convolutional neural network (GCN) has drawn increasing attention and attained good performance in various computer vision tasks, however, there lacks a clear interpretation of GCN's inner mechanism. For standard convolutional neural networks (CNNs), class activation mapping (CAM) methods are commonly used to visualize the connection between CNN's decision and image region by generating a heatmap. Nonetheless, such heatmap usually exhibits semantic-chaos when these CAMs are applied to GCN directly. In this paper, we proposed a novel visualization method particularly applicable to GCN, Vertex Semantic Class Activation Mapping (VS-CAM). VS-CAM includes two independent pipelines to produce a set of semantic-probe maps and a semantic-base map, respectively. Semantic-probe maps are used to detect the semantic information from semantic-base map to aggregate a semantic-aware heatmap. Qualitative results show that VS-CAM can obtain heatmaps where the highlighted regions match the objects much more precisely than CNN-based CAM. The quantitative evaluation further demonstrates the superiority of VS-CAM.

📄 PDF Abstract BibTeX arXiv:2209.09104

Code (0)

등록된 구현이 없습니다.

Tasks

Graph Neural Network

Methods 이 논문이 사용한 방법론

CAM Class activation maps could be used to interpret the prediction decision made by the convolutional neural network (CNN). Image source: [Learning Deep Features for…

Heatmap 설명 없음

GCN A Graph Convolutional Network, or GCN, is an approach for semi-supervised learning on graph-structured data. It is based on an efficient variant of [convolutional neural…

AGMN: Association Graph-based Graph Matching Network for Coronary Artery Semantic Labeling on Invasive Coronary Angiograms

2023-01-11 · Chen Zhao, Zhihui Xu, Jingfeng Jiang, Michele Esposito 외

Semantic labeling of coronary arterial segments in invasive coronary angiography (ICA) is important for automated assessment and report generation of coronary artery stenosis in the computer-aided diagnosis of coronary a…

Graph Matching

Towards Interpretable Semantic Segmentation via Gradient-weighted Class Activation Mapping

2020-02-26 · Kira Vinogradova, Alexandr Dibrov, Gene Myers

Convolutional neural networks have become state-of-the-art in a wide range of image recognition tasks. The interpretation of their predictions, however, is an active area of research. Whereas various interpretation metho…

image-classification Image Classification Image Segmentation Segmentation+1

Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attribution

2025-02-04 · Muhammad Umair Haider, Hammad Rizwan, Hassan Sajjad, Peizhong Ju 외

Interpreting the internal mechanisms of large language models (LLMs) is crucial for improving their trustworthiness and utility. Prior work has primarily focused on mapping individual neurons to discrete semantic concept…

Decoder text-classification Text Classification

TextCAM: Explaining Class Activation Map with Text

2025-10-01 · Qiming Zhao, Xingjian Li, Xiaoyu Cao, Xiaolong Wu 외 arxiv

Deep neural networks (DNNs) have achieved remarkable success across domains but remain difficult to interpret, limiting their trustworthiness in high-stakes applications. This paper focuses on deep vision models, for whi…

Disentangling concept semantics via multilingual averaging in Sparse Autoencoders

2025-08-19 · Cliff O'Reilly, Ernesto Jimenez-Ruiz, Tillman Weyde arxiv

Connecting LLMs with formal knowledge representation and reasoning is a promising approach to address their shortcomings. Embeddings and sparse autoencoders are widely used to represent textual content, but the semantics…