RECODE: Reasoning Through Code Generation for Visual Question Answering

2025-10-15 · Junhong Shen, Mu Cai, Bo Hu, Ameet Talwalkar, David A Ross, Cordelia Schmid, Alireza Fathi arxiv

Multimodal Large Language Models (MLLMs) struggle with precise reasoning for structured visuals like charts and diagrams, as pixel-based perception lacks a mechanism for verification. To address this, we propose to leverage derendering -- the process of reverse-engineering visuals into executable code -- as a new modality for verifiable visual reasoning. Specifically, we propose RECODE, an agentic framework that first generates multiple candidate programs to reproduce the input image. It then uses a critic to select the most faithful reconstruction and iteratively refines the code. This process not only transforms an ambiguous perceptual task into a verifiable, symbolic problem, but also enables precise calculations and logical inferences later on. On various visual reasoning benchmarks such as CharXiv, ChartQA, and Geometry3K, RECODE significantly outperforms methods that do not leverage code or only use code for drawing auxiliary lines or cropping. Our work demonstrates that grounding visual perception in executable code provides a new path toward more accurate and verifiable multimodal reasoning.

📄 PDF Abstract BibTeX arXiv:2510.13756

Code (0)

등록된 구현이 없습니다.

Tasks

Visual Question Answering Multimodal Reasoning Visual Reasoning Code Generation

ReCode: Reinforcing Code Generation with Reasoning-Process Rewards

2025-08-07 · Lishui Fan, Yu Zhang, Mouxiang Chen, Zhongxin Liu arxiv

In practice, rigorous reasoning is often a key driver of correct code, while Reinforcement Learning (RL) for code generation often neglects optimizing reasoning quality. Bringing process-level supervision into RL is appe…

Reinforcement Learning Code Generation

CORECODE: A Common Sense Annotated Dialogue Dataset with Benchmark Tasks for Chinese Large Language Models

2023-12-20 · Dan Shi, Chaobin You, Jiantao Huang, Taihao Li 외

As an indispensable ingredient of intelligence, commonsense reasoning is crucial for large language models (LLMs) in real-world scenarios. In this paper, we propose CORECODE, a dataset that contains abundant commonsense …

Causal Inference Common Sense Reasoning

ReCode: Updating Code API Knowledge with Reinforcement Learning

2025-06-25 · Haoze Wu, Yunzhi Yao, Wenhao Yu, Huajun Chen 외

Large Language Models (LLMs) exhibit remarkable code generation capabilities but falter when adapting to frequent updates in external library APIs. This critical limitation, stemming from reliance on outdated API knowled…

Code Generation reinforcement-learning Reinforcement Learning

MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correction

2026-04-02 · Zitian Tang, Xu Zhang, Jianbo Yuan, Yang Zou 외 arxiv

Multimodal Large Language Models (MLLMs) have recently demonstrated promising capabilities in multimodal coding tasks such as chart-to-code generation. However, existing methods primarily rely on supervised fine-tuning (…

Reinforcement Learning Code Generation

RECODE-H: A Benchmark for Research Code Development with Interactive Human Feedback

2025-10-07 · Chunyu Miao, Henry Peng Zou, Yangning Li, Yankai Chen 외 arxiv

Large language models (LLMs) show the promise in supporting scientific research implementation, yet their ability to generate correct and executable code remains limited. Existing works largely adopt one-shot settings, i…

Code Generation