Interpretable 2D Vision Models for 3D Medical Images
Training Artificial Intelligence (AI) models on 3D images presents unique challenges compared to the 2D case: Firstly, the demand for computational resources is significantly higher, and secondly, the availability of large datasets for pre-training is often limited, impeding training success. This study proposes a simple approach of adapting 2D networks with an intermediate feature representation for processing 3D images. Our method employs attention pooling to learn to assign each slice an importance weight and, by that, obtain a weighted average of all 2D slices. These weights directly quantify the contribution of each slice to the contribution and thus make the model prediction inspectable. We show on all 3D MedMNIST datasets as benchmark and two real-world datasets consisting of several hundred high-resolution CT or MRI scans that our approach performs on par with existing methods. Furthermore, we compare the in-built interpretability of our approach to HiResCam, a state-of-the-art retrospective interpretability approach.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
LocAnyMed: Vision-Language Grounding for Multimodal Medical Images
Medical visual grounding connects free-form clinical queries to spatial evidence in medical images and is an important component of interpretable medical artificial intelligence. However, general-purpose grounding models…
Visual GroundingHierarchical Vision Transformer with Prototypes for Interpretable Medical Image Classification
Explainability is a highly demanded requirement for applications in high-risk areas such as medicine. Vision Transformers have mainly been limited to attention extraction to provide insight into the model's reasoning. Ou…
image-classificationImage ClassificationLesion ClassificationMedical Image Classification+1Interpreting Latent Spaces of Generative Models for Medical Images using Unsupervised Methods
Generative models such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) play an increasingly important role in medical image analysis. The latent spaces of these models often show semanticall…
Medical Image AnalysisA Hybrid Fully Convolutional CNN-Transformer Model for Inherently Interpretable Medical Image Classification
In many medical imaging tasks, convolutional neural networks (CNNs) efficiently extract local features hierarchically. More recently, vision transformers (ViTs) have gained popularity, using self-attention mechanisms to …
image-classificationImage ClassificationMedical Image ClassificationAnatomy-VLM: A Fine-grained Vision-Language Model for Medical Interpretation
Accurate disease interpretation from radiology remains challenging due to imaging heterogeneity. Achieving expert-level diagnostic decisions requires integration of subtle image features with clinical knowledge. Yet majo…
Clinical KnowledgeImage Segmentation