홈 › Papers

JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzles

2026-07-30 · Shawn Li, Wei Yang, Jike Zhong, Jiate Li, Jiawei Yang, You Qin, Ryan Rossi, Franck Dernoncourt, Roger Zimmermann, Yue Wang, Zhengzhong Tu, Vicente Ordonez, Mohit Bansal, Yue Zhao arxiv

Jigsaw puzzle solving requires jointly reasoning about visual content and geometric constraints, yet existing benchmarks use rectangular cuts that create ambiguous ground truth in texture-repeated regions. We introduce \ours{}, a benchmark with tab-and-blank interlocking pieces where geometric constraints provide strong local compatibility requirements that, combined with visual content, yield unambiguous ground truth. Across 95K instances at four grid densities (4times4 to 16times16), we find that zero-shot VLMs largely lack geometric reasoning: only one of five frontier models (GPT-5.5) exceeds random baseline on 4times4 puzzles, while all others perform at chance level. While supervised fine-tuning achieves >97\% on 4times4, all models collapse on larger grids: GPT-5.5 drops from 70\% to near-random on 8times8, and even fine-tuned models fall below 5\% on 12times12. This ``scaling cliff'' suggests current architectures cannot maintain consistent constraint satisfaction as the number of pieces increases. establishes scalable geometric reasoning as an open challenge for vision-language models.

📄 PDF Abstract BibTeX arXiv:2607.27670

Code (0)

등록된 구현이 없습니다.

VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information

2024-12-01 · Ryo Kamoi, Yusen Zhang, Sarkar Snigdha Sarathi Das, Ranran Haoran Zhang 외

Large Vision Language Models (LVLMs) have achieved remarkable performance in various vision-language tasks. However, it is still unclear how accurately LVLMs can perceive visual information in images. In particular, the …

Multiple-choice

CompareBench: A Benchmark for Visual Comparison Reasoning in Vision-Language Models

2025-09-25 · Jie Cai, Kangning Yang, Lan Fu, Jiaming Ding 외 arxiv

We introduce CompareBench, a benchmark for evaluating visual comparison reasoning in vision-language models (VLMs), a fundamental yet understudied skill. CompareBench consists of 1000 QA pairs across four tasks: quantity…

Multimodal Reasoning

SPHINX: A Synthetic Environment for Visual Perception and Reasoning

2025-11-25 · Md Tanvirul Alam, Saksham Aggarwal, Justin Yang Chae, Nidhi Rastogi arxiv

We present Sphinx, a synthetic environment for visual perception and reasoning that targets core cognitive primitives. Sphinx procedurally generates puzzles using motifs, tiles, charts, icons, and geometric primitives, e…

Reinforcement Learning Multimodal Reasoning Symmetry Detection Spatial Reasoning

Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles

2026-01-20 · Maria Lymperaiou, Vasileios Karampinis, Giorgos Filandrianos, Angelos Vlachos 외 arxiv

Puzzles have long served as compact and revealing probes of human cognition, isolating abstraction, rule discovery, and systematic reasoning with minimal reliance on prior knowledge. Leveraging these properties, visual p…

Investigating Relational Reasoning in VLMs

2026-08-24 · Adhithya Laxman Ravi Shankar Geetha, Aulia Kharis Rakhmasari, Haleema Ramzan, Xander Yap arxiv

Vision-Language Models (VLMs) achieve strong performance in visual reasoning tasks, but it remains unclear whether they understand visual relations, or simply employ shortcuts such as language cues or priors. To investig…

Relational Reasoning Visual Reasoning