Do MLLMs See What We See? Analyzing Visualization Literacy Barriers in AI Systems
Multimodal Large Language Models (MLLMs) are increasingly used to interpret visualizations, yet little is known about why they fail. We present the first systematic analysis of barriers to visualization literacy in MLLMs. Using the regenerated Visualization Literacy Assessment Test (reVLAT) benchmark with synthetic data, we open-coded 309 erroneous responses from four state-of-the-art models with a barrier-centric strategy adapted from human visualization literacy research. Our analysis yields a taxonomy of MLLM failures, revealing two machine-specific barriers that extend prior human-participation frameworks. Results show that models perform well on simple charts but struggle with color-intensive, segment-based visualizations, often failing to form consistent comparative reasoning. Our findings inform future evaluation and design of reliable AI-driven visualization assistants.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Visualization Literacy of Multimodal Large Language Models: A Comparative Study
The recent introduction of multimodal large language models (MLLMs) combine the inherent power of large language models (LLMs) with the renewed capabilities to reason about the multimodal context. The potential usage sce…
Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
Multimodal large language models (MLLMs) are increasingly used to interpret visualizations, yet current evaluations remain largely chart-centric and provide limited evidence of understanding of scientific visualization (…
See or Recall: A Sanity Check for the Role of Vision in Solving Visualization Question Answer Tasks with Multimodal LLMs
Recent developments in multimodal large language models (MLLM) have equipped language models to reason about vision and language jointly. This permits MLLMs to both perceive and answer questions about data visualization …
Data VisualizationQuestion AnsweringTell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention
Accounting for individual differences can improve the effectiveness of visualization design. While the role of visual attention in visualization interpretation is well recognized, existing work often overlooks how this b…
Impact of Financial Literacy on Investment Decisions and Stock Market Participation using Extreme Learning Machines
The stock market has become an increasingly popular investment option among new generations, with individuals exploring more complex assets. This rise in retail investors' participation necessitates a deeper understandin…
Decision Makingfeature selectionTransfer Learning