paper-with-me

홈 › Papers

ChainV: Atomic Visual Hints Make Multimodal Reasoning Shorter and Better

2025-11-21 · Yuan Zhang, Ming Lu, Junwen Pan, Tao Huang, Kuan Cheng, Qi She, Shanghang Zhang arxiv

Recent advances in multimodal reasoning models have demonstrated impressive capabilities across text and vision. However, even leading models exhibit redundant self-reflection when generating lengthy reasoning chains. While training-free CoT compression methods have emerged in the LLMs domain, they rely on static visual references and thus provide limited gains for multimodal reasoning. Therefore, we propose ChainV, a framework that dynamically integrates visual hints into the reasoning process, thereby making multimodal reasoning shorter and better. Specifically, ChainV first performs a coarse visual patch selection based on the previous reasoning step, then refines it by identifying the most representative atomic visual hint according to the averaged attention intensity. Additionally, ChainV introduces a consistency-based evaluation mechanism to assess the reliability of the chosen hint, guiding the model to adaptively adjust its level of self-reflection. Eventually, the pixel coordinates of the selected visual hint and its reliability are incorporated into thinking with a Bernoulli stochastic process. Experiments indicate that our method significantly improves reasoning accuracy and efficiency, especially on math-intensive benchmarks where visual hints are crucial for multi-step symbolic reasoning. For example, ChainV achieves $2.3\%$ improvement on the MathVista within MIMO-VL-RL, while reducing inference latency by $51.4\%$ and shortening output token length by $24.5\%$.

📄 PDF Abstract BibTeX arXiv:2511.17106

Code (0)

등록된 구현이 없습니다.

Tasks

Multimodal Reasoning

Similar Papers 제목 키워드 기반

VisualHints: A Visual-Lingual Environment for Multimodal Reinforcement Learning

2020-10-26 · Thomas Carta, Subhajit Chaudhury, Kartik Talamadupula, Michiaki Tatsubori

We present VisualHints, a novel environment for multimodal reinforcement learning (RL) involving text-based interactions along with visual hints (obtained from the environment). Real-life problems often demand that agent…

Atari Gamesreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1

Hints of Prompt: Enhancing Visual Representation for Multimodal LLMs in Autonomous Driving

2024-11-20 · Hao Zhou, Zhanning Gao, Maosheng Ye, Zhili Chen 외

In light of the dynamic nature of autonomous driving environments and stringent safety requirements, general MLLMs combined with CLIP alone often struggle to represent driving-specific scenarios accurately, particularly …

Autonomous DrivingMultimodal ReasoningVisual Question Answering (VQA)

ChainVLA: Chaining Vision-Language-Action Queries through a Unified Execution State for Long-Horizon Manipulation

2026-08-03 · Yuzhi Huang, Weijue Bu, Ziyi Xiong, Jie Wu 외 arxiv

Humans perform long-horizon manipulation by retaining knowledge of what earlier actions have established while continuously adapting the motion underway. By contrast, action-chunked vision-language-action (VLA) policies …

Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering

2019-04-08 · CVPR 2019 6 · Chenyou Fan, Xiaofan Zhang, Shu Zhang, Wensheng Wang 외

In this paper, we propose a novel end-to-end trainable Video Question Answering (VideoQA) framework with three major components: 1) a new heterogeneous memory which can effectively learn global context information from a…

Question AnsweringVideo Question AnsweringVisual Question Answering (VQA)

HintsOfTruth: A Multimodal Checkworthiness Detection Dataset with Real and Synthetic Claims

2025-02-17 · Michiel van der Meer, Pavel Korshunov, Sébastien Marcel, Lonneke van der Plas

Misinformation can be countered with fact-checking, but the process is costly and slow. Identifying checkworthy claims is the first step, where automation can help scale fact-checkers' efforts. However, detection methods…

BenchmarkingFact CheckingMisinformation