ProtoMask: Segmentation-Guided Prototype Learning
XAI gained considerable importance in recent years. Methods based on prototypical case-based reasoning have shown a promising improvement in explainability. However, these methods typically rely on additional post-hoc saliency techniques to explain the semantics of learned prototypes. Multiple critiques have been raised about the reliability and quality of such techniques. For this reason, we study the use of prominent image segmentation foundation models to improve the truthfulness of the mapping between embedding and input space. We aim to restrict the computation area of the saliency map to a predefined semantic image patch to reduce the uncertainty of such visualizations. To perceive the information of an entire image, we use the bounding box from each generated segmentation mask to crop the image. Each mask results in an individual input in our novel model architecture named ProtoMask. We conduct experiments on three popular fine-grained classification datasets with a wide set of metrics, providing a detailed overview on explainability characteristics. The comparison with other popular models demonstrates competitive performance and unique explainability features of our model. https://github.com/uos-sis/quanproto
Code (0)
등록된 구현이 없습니다.
Tasks
Image SegmentationSimilar Papers 제목 키워드 기반
Adaptive Prototype Learning and Allocation for Few-Shot Segmentation
Prototype learning is extensively used for few-shot segmentation. Typically, a single prototype is obtained from the support feature by averaging the global object information. However, using one prototype to represent a…
ClusteringFew-Shot Semantic SegmentationSegmentationPrototype Guided Network for Anomaly Segmentation
Semantic segmentation methods can not directly identify abnormal objects in images. Anomaly Segmentation algorithm from this realistic setting can distinguish between in-distribution objects and Out-Of-Distribution (OOD)…
Anomaly SegmentationSegmentationSemantic SegmentationAlleviating Textual Reliance in Medical Language-guided Segmentation via Prototype-driven Semantic Approximation
Medical language-guided segmentation, integrating textual clinical reports as auxiliary guidance to enhance image segmentation, has demonstrated significant improvements over unimodal approaches. However, its inherent re…
Image SegmentationSegmentationSemantic SegmentationQuery-guided Prototype Evolution Network for Few-Shot Segmentation
Previous Few-Shot Segmentation (FSS) approaches exclusively utilize support features for prototype generation, neglecting the specific requirements of the query. To address this, we present the Query-guided Prototype Evo…
SegmentationSPENet: Self-guided Prototype Enhancement Network for Few-shot Medical Image Segmentation
Few-Shot Medical Image Segmentation (FSMIS) aims to segment novel classes of medical objects using only a few labeled images. Prototype-based methods have made significant progress in addressing FSMIS. However, they typi…
Medical Image Segmentation