FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts
Existing benchmarks for visual question answering lack in visual grounding and complexity, particularly in evaluating spatial reasoning skills. We introduce FlowVQA, a novel benchmark aimed at assessing the capabilities of visual question-answering multimodal language models in reasoning with flowcharts as visual contexts. FlowVQA comprises 2,272 carefully generated and human-verified flowchart images from three distinct content sources, along with 22,413 diverse question-answer pairs, to test a spectrum of reasoning tasks, including information localization, decision-making, and logical progression. We conduct a thorough baseline evaluation on a suite of both open-source and proprietary multimodal language models using various strategies, followed by an analysis of directional bias. The results underscore the benchmark's potential as a vital tool for advancing the field of multimodal modeling, providing a focused and challenging environment for enhancing model performance in visual and logical reasoning tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingLogical ReasoningQuestion AnsweringSpatial ReasoningVisual GroundingVisual Question AnsweringSimilar Papers 제목 키워드 기반
StackOverflowVQA: Stack Overflow Visual Question Answering Dataset
In recent years, people have increasingly used AI to help them with their problems by asking questions on different topics. One of these topics can be software-related and programming questions. In this work, we focus on…
Question AnsweringSentenceVisual Question AnsweringVisual Question Answering (VQA)LININ: Logic Integrated Neural Inference Network for Explanatory Visual Question Answering
Explanatory Visual Question Answering (EVQA) is a recently proposed multimodal reasoning task consisting of answering the visual question and generating multimodal explanations for the reasoning processes. Unlike traditi…
Explanatory Visual Question AnsweringMultimodal ReasoningQuestion AnsweringVisual Question Answering+1LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts
We propose LogicVista, an evaluation benchmark that assesses the integrated logical reasoning capabilities of multimodal large language models (MLLMs) in Visual contexts. Recent advancements in MLLMs have demonstrated va…
Logical ReasoningMathematical ReasoningMultiple-choiceAdapting Visual Question Answering Models for Enhancing Multimodal Community Q&A Platforms
Question categorization and expert retrieval methods have been crucial for information organization and accessibility in community question & answering (CQA) platforms. Research in this area, however, has dealt with only…
Community Question AnsweringGeneral ClassificationQuestion AnsweringRepresentation Learning+4Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images
Survival outcome prediction is a challenging weakly-supervised and ordinal regression task in computational pathology that involves modeling complex interactions within the tumor microenvironment in gigapixel whole s…
AttributeMultiple Instance LearningQuestion AnsweringRepresentation Learning+5