TraversalBench: Challenging Paths to Follow for Vision Language Models
Vision-language models (VLMs) perform strongly on multimodal benchmarks, but their ability to follow complex visual paths remains under-tested. We introduce TraversalBench, a controlled benchmark for exact visual path traversal. Each instance contains a continuous polyline with a unique start marker and labeled vertices; models must recover the ordered sequence encountered from start to finish. The benchmark balances self-intersection count, tortuosity, vertex count, and nearby confounding lines while limiting reliance on OCR, world knowledge, or open-ended planning. We find that self-intersections are the dominant source of difficulty. A first-crossing analysis localizes failures to crossing points: performance is stable before the first crossing, then drops sharply when the model must resolve the correct continuation. Nearby confounders have weaker but compounding effects, and an auxiliary reading-order benchmark reveals a consistent left-to-right bias. Together, these results characterize how VLMs perceive and fail on visual paths. Finally, we position TraversalBench as a new contribution to the growing line of sustained and precise visual grounding benchmarks for VLMs. Code, benchmark data, and rendered examples are available at https://github.com/clarapetrova/traversalbench.
Code (0)
등록된 구현이 없습니다.
Tasks
Visual GroundingSimilar Papers 제목 키워드 기반
GameTraversalBenchmark: Evaluating Planning Abilities Of Large Language Models Through Traversing 2D Game Maps
Large language models (LLMs) have recently demonstrated great success in generating and understanding natural language. While they have also shown potential beyond the domain of natural language, it remains an open quest…
Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
Advances in learning and representations have reinvigorated work that connects language to other modalities. A particularly exciting direction is Vision-and-Language Navigation(VLN), in which agents interpret natural lan…
Instruction FollowingVision and Language NavigationReinforcing Dual-Path Reasoning in Spatial Vision Language Models
Spatial VLMs have made substantial progress in geometric perception, yet complex spatial reasoning requiring multi-step inference over depth, distance, and scene relations remains challenging. Moreover, different spatial…
Reinforcement LearningSpatial ReasoningV-REX: Benchmarking Exploratory Visual Reasoning via Chain-of-Questions
While many vision-language models (VLMs) are developed to answer well-defined, straightforward questions with highly specified targets, as in most benchmarks, they often struggle in practice with complex open-ended tasks…
Visual ReasoningPathISE: Learning Informative Path Supervision for Knowledge Graph Question Answering
Knowledge Graph Question Answering (KGQA) aims to answer user questions by reasoning over Knowledge Graphs (KGs). Recent KGQA methods mainly follow the retrieval-augmented generation paradigm to ground Large Language Mod…
Graph Question AnsweringKnowledge Graphs