Correcting What You Cannot See: Credit Assignment for Perception Distillation in Multimodal Reasoners

2026-07-30 · Feng Xiong, Leyan Xue, Hongyu Lin arxiv

On-policy distillation provides dense supervision for multimodal reasoners, but its trajectory-level reward cannot determine whether a failed answer arose from perception or subsequent reasoning. Perception Success Rate (PSR), estimated from multiple reasonings sharing one perception, remains ambiguous because low success conflates perceptual insufficiency with reasoning difficulty. We introduce \textbf{Perception-Correction Distillation (PCD)}, a label-free method that identifies correctable perception failures using downstream failure and teacher--student disagreement as complementary witnesses. Their product, , forms a soft AND gate that strengthens distillation only when both witnesses are present. We motivate this rule through Bayesian evidence combination and show that multiplication is the unique normalized bilinear gate that vanishes when either witness is absent. PCD uses separated perception--reasoning rollouts and mean-preserving weights, leaving the reasoning objective unchanged. Across eight benchmarks, PCD improves the 8B 2B macro average from 44.50 with OPD to 47.28 and the 32B 8B result from 56.94 to 61.22. In matched 2B ablations, removing PCD and separated rollout reduces held-out average by 2.22 and 0.88 points, respectively. Effective multimodal distillation therefore depends not only on what the teacher predicts, but also on identifying when perception is the appropriate target of correction.

📄 PDF Abstract BibTeX arXiv:2607.28336

Code (0)

등록된 구현이 없습니다.

Hindsight Credit Assignment

2019-12-05 · NeurIPS 2019 12 · Anna Harutyunyan, Will Dabney, Thomas Mesnard, Mohammad Azar 외

We consider the problem of efficient credit assignment in reinforcement learning. In order to efficiently and meaningfully utilize new data, we propose to explicitly assign credit to past decisions based on the likelihoo…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

RACA: Relation-Aware Credit Assignment for Ad-Hoc Cooperation in Multi-Agent Deep Reinforcement Learning

2022-06-02 · Hao Chen, Guangkai Yang, Junge Zhang, Qiyue Yin 외

In recent years, reinforcement learning has faced several challenges in the multi-agent domain, such as the credit assignment issue. Value function factorization emerges as a promising way to handle the credit assignment…

Deep Reinforcement Learning Reinforcement Learning (RL)Relation Zero-shot Generalization

Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents?

2026-05-27 · Yibo Zhao, Zichen Ding, Jiayi Wu, Zun Wang 외 arxiv

Search agents powered by large language models can autonomously decompose queries, retrieve information, and synthesize answers through multi-step reasoning. However, the rapid growth of training methods has outpaced con…

Learning Explicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning via Polarization Policy Gradient

2022-10-10 · Wubing Chen, Wenbin Li, Xiao Liu, Shangdong Yang 외

Cooperative multi-agent policy gradient (MAPG) algorithms have recently attracted wide attention and are regarded as a general scheme for the multi-agent system. Credit assignment plays an important role in MAPG and can …

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Starcraft+1

TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning

2026-06-30 · Yuanda Xu, Zhengze Zhou, Hejian Sang, Xiaomin Li 외 hf

Agentic reinforcement learning requires assigning credit to environment-facing actions such as searches, clicks, edits, navigation commands, and object interactions. Standard GRPO uses the final verifier outcome as a uni…

Reinforcement Learning