paper-with-me

홈 › Papers

Envisioning MedCLIP: A Deep Dive into Explainability for Medical Vision-Language Models

2024-03-27 · Anees Ur Rehman Hashmi, Dwarikanath Mahapatra, Mohammad Yaqub

Explaining Deep Learning models is becoming increasingly important in the face of daily emerging multimodal models, particularly in safety-critical domains like medical imaging. However, the lack of detailed investigations into the performance of explainability methods on these models is widening the gap between their development and safe deployment. In this work, we analyze the performance of various explainable AI methods on a vision-language model, MedCLIP, to demystify its inner workings. We also provide a simple methodology to overcome the shortcomings of these methods. Our work offers a different new perspective on the explainability of a recent well-known VLM in the medical domain and our assessment method is generalizable to other current and possible future VLMs.

📄 PDF Abstract BibTeX arXiv:2403.18996

Code (0)

등록된 구현이 없습니다.

Tasks

Language ModelingLanguage Modelling

Similar Papers 제목 키워드 기반

BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs

2023-03-02 · Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu 외

Biomedical data is inherently multimodal, comprising physical measurements and natural language narratives. A generalist biomedical AI model needs to simultaneously process different modalities of data, including text an…

ArticlesMedical Visual Question AnsweringPneumonia DetectionQuestion Answering+3

MedCLIPSeg: Probabilistic Vision-Language Adaptation for Data-Efficient and Generalizable Medical Image Segmentation

2026-02-23 · Taha Koleilat, Hojat Asgariandehkordi, Omid Nejati Manzari, Berardino Barile 외 arxiv

Medical image segmentation remains challenging due to limited annotations for training, ambiguous anatomical features, and domain shifts. While vision-language models such as CLIP offer strong cross-modal representations…

Medical Image Segmentation

Backdoor Attack on Unpaired Medical Image-Text Foundation Models: A Pilot Study on MedCLIP

2024-01-01 · Ruinan Jin, Chun-Yin Huang, Chenyu You, Xiaoxiao Li

In recent years, foundation models (FMs) have solidified their role as cornerstone advancements in the deep learning domain. By extracting intricate patterns from vast datasets, these models consistently achieve state-of…

Backdoor AttackContrastive LearningImage-text matchingText Matching

Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain?

2021-12-27 · Sedigheh Eslami, Gerard de Melo, Christoph Meinel

Contrastive Language--Image Pre-training (CLIP) has shown remarkable success in learning with cross-modal supervision from extensive amounts of image--text pairs collected online. Thus far, the effectiveness of CLIP has …

ArticlesMedical Visual Question AnsweringMeta-LearningQuestion Answering+3

MedCLIP-SAM: Bridging Text and Image Towards Universal Medical Image Segmentation

2024-03-29 · Taha Koleilat, Hojat Asgariandehkordi, Hassan Rivaz, Yiming Xiao

Medical image segmentation of anatomical structures and pathology is crucial in modern clinical diagnosis, disease study, and treatment planning. To date, great progress has been made in deep learning-based segmentation …

Image SegmentationMedical Image AnalysisMedical Image SegmentationSegmentation+2