Reasoning-Guided Part-Level Visual Grounding via Reinforcement Learning
Multimodal large language models (MLLMs) ground whole objects well from free-form language queries, but they struggle when the query names a part rather than the object. We trace this to a missing object-part hierarchy, since parts are localized in the same single step used for objects. We propose Object-Part Hierarchical Reflective Grounding (OP-HRG), a coarse-to-fine reasoning-guided grounding strategy that first localizes the parent object and then the part within it. A self-check then reflects on the result, with an extension to re-encode the predicted crop to inspect the region it is correcting. We introduce a part-aware GRPO framework to train our pipeline with stage-wise rewards. A 4B model trained this way outperforms 7B grounding LLMs and SAM3 across PascalPart, PartImageNet, and InstructPart, and transfers to reasoning segmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningVisual GroundingSimilar Papers 제목 키워드 기반
UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning
Traditional visual grounding methods primarily focus on single-image scenarios with simple textual references. However, extending these methods to real-world scenarios that involve implicit and complex instructions, part…
Large Language ModelMultimodal Large Language ModelReinforcement Learning (RL)Visual GroundingREVEALER: Reinforcement-Guided Visual Reasoning for Element-Level Text-Image Alignment Evaluation
Evaluating the alignment between textual prompts and generated images is critical for ensuring the reliability and usability of text-to-image (T2I) models. However, most existing evaluation methods rely on coarse-grained…
Visual ReasoningTowards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition
Understanding modality interaction in multimodal large language models (MLLMs) is central to reliable deployment. We introduce Partial Information Decomposition (PID) as a decision-level framework that separates unique, …
Multimodal ReasoningBeyond Static Cropping: Layer-Adaptive Visual Localization and Decoding Enhancement
Large Vision-Language Models (LVLMs) have advanced rapidly by aligning visual patches with the text embedding space, but a fixed visual-token budget forces images to be resized to a uniform pretraining resolution, often …
Visual LocalizationQuestion AnsweringObject RecognitionVisual GroundingBeyond Where to Look: Trajectory-Guided Reinforcement Learning for Multimodal RLVR
Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) for multimodal large language models (MLLMs) have mainly focused on improving final answer correctness and strengthening visual grounding. However,…
Reinforcement LearningMultimodal ReasoningLogical ReasoningVisual Grounding