A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts
Despite substantial progress in applying neural networks (NN) to a wide variety of areas, they still largely suffer from a lack of transparency and interpretability. While recent developments in explainable artificial intelligence attempt to bridge this gap (e.g., by visualizing the correlation between input pixels and final outputs), these approaches are limited to explaining low-level relationships, and crucially, do not provide insights on error correction. In this work, we propose a framework (VRX) to interpret classification NNs with intuitive structural visual concepts. Given a trained classification model, the proposed VRX extracts relevant class-specific visual concepts and organizes them using structural concept graphs (SCG) based on pairwise concept relationships. By means of knowledge distillation, we show VRX can take a step towards mimicking the reasoning process of NNs and provide logical, concept-level explanations for final model decisions. With extensive experiments, we empirically show VRX can meaningfully answer "why" and "why not" questions about the prediction, providing easy-to-understand insights about the reasoning process. We also show that these insights can potentially provide guidance on improving NN's performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Explainable artificial intelligenceKnowledge DistillationSimilar Papers 제목 키워드 기반
PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policies
Robotic manipulation policies often fail to generalize because they must simultaneously learn where to attend, what actions to take, and how to execute them. We argue that high-level reasoning about where and what can be…
Zero-shot GeneralizationRobot ManipulationPEEK: Picking Essential frames via Efficient Knowledge distillation
Video-language models can process only a limited number of frames, making frame selection a key bottleneck for efficient video captioning. Most captioning pipelines still rely on uniform sampling, which is computationall…
Knowledge DistillationVideo CaptioningPEEK: Context Map as an Orientation Cache for Long-Context LLM Agents
Large language model (LLM) agents increasingly operate over long and recurring external contexts, like document corpora and code repositories. Across invocations, existing approaches preserve either the agent's trajector…
Peek-a-Boo Reasoning: Contrastive Region Masking in MLLMs
We introduce Contrastive Region Masking (CRM), a training free diagnostic that reveals how multimodal large language models (MLLMs) depend on specific visual regions at each step of chain-of-thought (CoT) reasoning. Unli…
A new promising material for biological applications: multi-level physical modification of AgNPs-decorated PEEK
In the case of polymer medical devices, the surface design plays a crucial role in contact with human tissue. The use of AgNPs as antibacterial agents is well known; however, their anchoring into the polymer surface can …