Case-Enhanced Vision Transformer: Improving Explanations of Image Similarity with a ViT-based Similarity Metric

2024-07-24 · Ziwei Zhao, David Leake, Xiaomeng Ye, David Crandall

This short paper presents preliminary research on the Case-Enhanced Vision Transformer (CEViT), a similarity measurement method aimed at improving the explainability of similarity assessments for image data. Initial experimental results suggest that integrating CEViT into k-Nearest Neighbor (k-NN) classification yields classification accuracy comparable to state-of-the-art computer vision models, while adding capabilities for illustrating differences between classes. CEViT explanations can be influenced by prior cases, to illustrate aspects of similarity relevant to those cases.

📄 PDF Abstract BibTeX arXiv:2407.16981

Code (1)

ziweizhao1993/cevit 공식 구현 pytorch

Tasks

Classification

Methods 이 논문이 사용한 방법론

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

Position-Wise Feed-Forward Layer 설명 없음

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Towards Evaluating Explanations of Vision Transformers for Medical Imaging

2023-04-12 · Piotr Komorowski, Hubert Baniecki, Przemysław Biecek

As deep learning models increasingly find applications in critical domains such as medical imaging, the need for transparent and trustworthy decision-making becomes paramount. Many explainability methods provide insights…

Decision Making image-classification Image Classification

Attribution for Enhanced Explanation with Transferable Adversarial eXploration

2024-12-27 · Zhiyu Zhu, Jiayu Zhang, Zhibo Jin, Huaming Chen 외

The interpretability of deep neural networks is crucial for understanding model decisions in various applications, including computer vision. AttEXplore++, an advanced framework built upon AttEXplore, enhances attributio…

Adversarial Attack Diversity

UbiQVision: Quantifying Uncertainty in XAI for Image Recognition

2025-12-23 · Akshat Dubey, Aleksandar Anžel, Bahar İlgen, Georges Hattab arxiv

Recent advances in deep learning have led to its widespread adoption across diverse domains, including medical imaging. This progress is driven by increasingly sophisticated model architectures, such as ResNets, Vision T…

V-CECE: Visual Counterfactual Explanations via Conceptual Edits

2025-09-20 · Nikolaos Spanos, Maria Lymperaiou, Giorgos Filandrianos, Konstantinos Thomas 외 arxiv

Recent black-box counterfactual generation frameworks fail to take into account the semantic content of the proposed edits, while relying heavily on training to guide the generation process. We propose a novel, plug-and-…

Image Editing

Interpretable Image Classification with Adaptive Prototype-based Vision Transformers

2024-10-28 · Chiyu Ma, Jon Donnelly, Wenjun Liu, Soroush Vosoughi 외

We present ProtoViT, a method for interpretable image classification combining deep learning and case-based reasoning. This method classifies an image by comparing it to a set of learned prototypes, providing explanation…

image-classification Image Classification