MJ1: Multimodal Judgment via Grounded Verification

2026-03-09 · Bhavesh Kumar, Dylan Feng, Leonard Tang arxiv

Multimodal judges struggle to ground decisions in visual evidence. We present MJ1, a multimodal judge trained with reinforcement learning that enforces visual grounding through a structured grounded verification chain (observations $\rightarrow$ claims $\rightarrow$ verification $\rightarrow$ evaluation $\rightarrow$ scoring) and a counterfactual consistency reward that penalizes position bias. Even without training, our mechanism improves base-model accuracy on MMRB2 by +3.8 points on Image Editing and +1.7 on Multimodal Reasoning. After training, MJ1, with only 3B active parameters, achieves 77.0% accuracy on MMRB2 and surpasses orders-of-magnitude larger models like Gemini-3-Pro. These results show that grounded verification and consistency-based training substantially improve multimodal judgment without increasing model scale.

📄 PDF Abstract BibTeX arXiv:2603.07990

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Multimodal Reasoning Visual Grounding Image Editing

DualFact+: A Multimodal Fact Verification Framework for Procedural Video Understanding

2026-04-28 · Cennet Oguz, Yasser Hamidullah, Josef van Genabith, Simon Ostermann arxiv

We introduce DualFact, a dual-layer, multimodal factuality evaluation framework for procedural video captioning. DualFact separates factual correctness into conceptual facts, capturing abstract semantic roles (e.g., Acti…

Fact Verification Video Captioning

SciLens: Multi-modal Scientific Claim Verification with Agentic Entailment and Grounding

2026-06-18 · Yueming Wang, Tianshi Zheng, Jiaxin Bai, Yangqiu Song 외 arxiv

Scientific discovery increasingly relies on automated systems that generate hypotheses, inspect multimodal evidence, and validate claims at scale. Yet scientific claim verification is not well served by asking a vision-l…

DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification

2026-05-10 · Rui Liu, Dian Yu, Zhenwen Liang, Yucheng Shi 외 arxiv

Aligning Multimodal Large Language Models (MLLMs) requires reliable reward models, yet existing single-step evaluators can suffer from lazy judging, exploiting language priors over fine-grained visual verification. While…

Reinforcement Learning Visual Reasoning

MERIT: Modular Framework for Multimodal Misinformation Detection with Web-Grounded Reasoning

2025-10-20 · Mir Nafis Sharear Shopnil, Sharad Duwal, Abhishek Tyagi, Adiba Mahbub Proma arxiv

We present MERIT, an inference-time modular framework for multimodal misinformation detection that decomposes verification into four specialized modules: visual forensics, cross-modal alignment, retrieval-augmented claim…

Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms

2023-10-16 · Seungju Han, Junhyeok Kim, Jack Hessel, Liwei Jiang 외

Commonsense norms are defeasible by context: reading books is usually great, but not when driving a car. While contexts can be explicitly described in language, in embodied scenarios, contexts are often provided visually…