StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning

2026-05-12 · Hao Wang, Rui Li, Lei Sha, Jie M. Zhang arxiv

Existing code reasoning methods primarily supervise final code outputs, ignoring intermediate states, often leading to reward hacking where correct answers are obtained through inconsistent reasoning. We propose StepCodeReasoner, a framework that introduces explicit intermediate execution-state supervision. By automatically inserting structured print-based execution-trace anchors into code, the model is trained to predict runtime states at each step, transforming code reasoning into a verifiable, stepwise execution modeling problem. Building on this execution-aware method, we introduce Bi-Level GRPO, a reinforcement learning algorithm for structured credit assignment at two levels: inter-trajectory, comparing alternative execution paths, and intra-trajectory, rewarding intermediate accuracy based on its impact on downstream correctness. Extensive experiments demonstrate that StepCodeReasoner achieves SOTA performance in code reasoning. In particular, our 7B model achieves 91.1\% on CRUXEval and 86.5\% on LiveCodeBench, outperforming the CodeReasoner-7B baseline (86.0\% and 77.7\%) and GPT-4o (85.6\% and 75.1\%). Furthermore, on the execution-trace benchmark REval, our model scores 82.9\%, outperforming baseline CodeReasoner-7B (72.3\%), its 14B counterpart (81.1\%), and GPT-4o (77.3\%). Additionally, our approach also improves code generation performance, demonstrating that explicit execution modeling enhances both code reasoning and code generation.

📄 PDF Abstract BibTeX arXiv:2605.11922

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Code Generation

Stepwise Reasoning Error Disruption Attack of LLMs

2024-12-16 · Jingyu Peng, Maolin Wang, Xiangyu Zhao, Kai Zhang 외

Large language models (LLMs) have made remarkable strides in complex reasoning tasks, but their safety and robustness in reasoning processes remain underexplored. Existing attacks on LLM reasoning are constrained by spec…

StepPRM-RTL: Stepwise Process-Reward Guided LLM Fine-Tuning for Enhanced RTL Synthesis

2026-06-02 · Prashanth Vijayaraghavan, Apoorva Nitsure, Luyao Shi, Ehsan Degan 외 arxiv

Automatic generation of RTL code for digital hardware designs remains challenging due to long-horizon reasoning, multi-step dependencies, and strict correctness constraints in Verilog and VHDL. We present StepPRM-RTL, a …

Trajectory Modeling Code Generation

Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution

2026-05-19 · Xiaoou Liu, Tiejin Chen, Dengjia Zhang, Yaqing Wang 외 arxiv

Large Language Models have achieved strong performance on reasoning tasks with objective answers by generating step-by-step solutions, but diagnosing where a multi-step reasoning trace might fail remains difficult. Confi…

Multi-hop Question Answering Mathematical Reasoning

ACE-TA: An Agentic Teaching Assistant for Grounded Q&A, Quiz Generation, and Code Tutoring

2026-02-20 · Himanshu Tripathi, Charlottee Crowell, Kaley Newlin, Subash Neupane 외 arxiv

We introduce ACE-TA, the Agentic Coding and Explanations Teaching Assistant framework, that autonomously routes conceptual queries drawn from programming course material to grounded Q&A, stepwise coding guidance, and aut…

ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQL

2025-05-19 · Yaxun Dai, Wenxuan Xie, Xialie Zhuang, Tianyu Yang 외

In Text-to-SQL, execution feedback is essential for guiding large language models (LLMs) to reason accurately and generate reliable SQL queries. However, existing methods treat execution feedback solely as a post-hoc sig…

Text to SQL Text-To-SQL