Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking
Evidence data for automated fact-checking (AFC) can be in multiple modalities such as text, tables, images, audio, or video. While there is increasing interest in using images for AFC, previous works mostly focus on detecting manipulated or fake images. We propose a novel task, chart-based fact-checking, and introduce ChartBERT as the first model for AFC against chart evidence. ChartBERT leverages textual, structural and visual information of charts to determine the veracity of textual claims. For evaluation, we create ChartFC, a new dataset of 15, 886 charts. We systematically evaluate 75 different vision-language (VL) baselines and show that ChartBERT outperforms VL models, achieving 63.8% accuracy. Our results suggest that the task is complex yet feasible, with many challenges ahead.
Code (0)
등록된 구현이 없습니다.
Tasks
Fact CheckingSimilar Papers 제목 키워드 기반
ChartProbe: A Diagnostic Study on Visual Reasoning through Perception, Grounding, and Simple Reasoning
Vision-language models (VLMs) remain unreliable on chart questions that require reasoning over visual quantities, and this weakness is usually attributed to a reasoning deficit and addressed with more reasoning supervisi…
Visual ReasoningMultimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model
Although most current large multimodal models (LMMs) can already understand photos of natural scenes and portraits, their understanding of abstract images, e.g., charts, maps, or layouts, and visual reasoning capabilitie…
Chart UnderstandingLanguage ModelingLanguage ModellingVisual ReasoningTensor Fields for Data Extraction from Chart Images: Bar Charts and Scatter Plots
Charts are an essential part of both graphicacy (graphical literacy), and statistical literacy. As chart understanding has become increasingly relevant in data science, automating chart analysis by processing raster imag…
Chart UnderstandingViDR: Grounding Multimodal Deep Research Reports in Source Visual Evidence
Recent deep research systems have improved the ability of large language models to produce long, grounded reports through iterative retrieval and reasoning. However, most text-centered systems rely mainly on textual evid…
DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?
Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities, and rendering them accurately. Modern LLMs can produce visually pl…