Explaining Multi-modal Large Language Models by Analyzing their Vision Perception
Multi-modal Large Language Models (MLLMs) have demonstrated remarkable capabilities in understanding and generating content across various modalities, such as images and text. However, their interpretability remains a challenge, hindering their adoption in critical applications. This research proposes a novel approach to enhance the interpretability of MLLMs by focusing on the image embedding component. We combine an open-world localization model with a MLLM, thus creating a new architecture able to simultaneously produce text and object localization outputs from the same vision embedding. The proposed architecture greatly promotes interpretability, enabling us to design a novel saliency map to explain any output token, to identify model hallucinations, and to assess model biases through semantic adversarial perturbations.
Code (1)
Tasks
Object LocalizationSimilar Papers 제목 키워드 기반
Few-Shot Multimodal Explanation for Visual Question Answering
A key object in eXplainable Artificial Intelligence (XAI) is to create intelligent systems capable of reasoning and explaining real-world data to facilitate reliable decision-making. Recent studies have acknowledged the …
Explainable artificial intelligenceExplainable Artificial Intelligence (XAI)FS-MEVQAQuestion Answering+3Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis
We present a novel framework for analyzing and interpreting electron microscopy images in semiconductor manufacturing using vision-language instruction tuning. The framework employs a unique teacher-student approach, lev…
Instruction FollowingQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis
Multimodal sentiment analysis aims to recognize people's attitudes from multiple communication channels such as verbal content (i.e., text), voice, and facial expressions. It has become a vibrant and important research t…
Multimodal Sentiment AnalysisSentiment AnalysisDecoding the Underlying Meaning of Multimodal Hateful Memes
Recent studies have proposed models that yielded promising performance for the hateful meme classification task. Nevertheless, these proposed models do not generate interpretable explanations that uncover the underlying …
BenchmarkingHateful Meme ClassificationMeme ClassificationWhere MLLMs Attend and What They Rely On: Explaining Autoregressive Token Generation
Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in aligning visual inputs with natural language outputs. Yet, the extent to which generated tokens depend on visual modalities remains po…