홈 › Papers

StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback

2024-02-02 · Shihan Dou, Yan Liu, Haoxiang Jia, Limao Xiong, Enyu Zhou, Wei Shen, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Xuanjing Huang, Tao Gui

The advancement of large language models (LLMs) has significantly propelled the field of code generation. Previous work integrated reinforcement learning (RL) with compiler feedback for exploring the output space of LLMs to enhance code generation quality. However, the lengthy code generated by LLMs in response to complex human requirements makes RL exploration a challenge. Also, since the unit tests may not cover the complicated code, optimizing LLMs by using these unexecuted code snippets is ineffective. To tackle these challenges, we introduce StepCoder, a novel RL framework for code generation, consisting of two main components: CCCS addresses the exploration challenge by breaking the long sequences code generation task into a Curriculum of Code Completion Subtasks, while FGO only optimizes the model by masking the unexecuted code segments to provide Fine-Grained Optimization. In addition, we furthermore construct the APPS+ dataset for RL training, which is manually verified to ensure the correctness of unit tests. Experimental results show that our method improves the ability to explore the output space and outperforms state-of-the-art approaches in corresponding benchmarks. Our dataset APPS+ and StepCoder are available online.

📄 PDF Abstract BibTeX arXiv:2402.01391

Code (1)

ablustrund/apps_plus 공식 구현

Tasks

Code Completion Code Generation reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning

2026-05-12 · Hao Wang, Rui Li, Lei Sha, Jie M. Zhang arxiv

Existing code reasoning methods primarily supervise final code outputs, ignoring intermediate states, often leading to reward hacking where correct answers are obtained through inconsistent reasoning. We propose StepCode…

Reinforcement Learning Code Generation

Compilable Neural Code Generation with Compiler Feedback

2022-03-10 · Findings (ACL) 2022 5 · Xin Wang, Yasheng Wang, Yao Wan, Fei Mi 외

Automatically generating compilable programs with (or without) natural language descriptions has always been a touchstone problem for computational linguistics and automated software engineering. Existing deep-learning a…

Code Completion Code Generation Decoder Language Modeling+3

AutoPhase: Compiler Phase-Ordering for High Level Synthesis with Deep Reinforcement Learning

2019-01-15 · Ameer Haj-Ali, Qijing Huang, William Moses, John Xiang 외

The performance of the code generated by a compiler depends on the order in which the optimization passes are applied. In high-level synthesis, the quality of the generated circuit relates directly to the code generated …

Deep Reinforcement Learning High-Level Synthesis reinforcement-learning Reinforcement Learning+1

Generating GPU Compiler Heuristics using Reinforcement Learning

2021-11-23 · Ian Colbert, Jake Daly, Norm Rubin

GPU compilers are complex software programs with many optimizations specific to target hardware. These optimizations are often controlled by heuristics hand-designed by compiler experts using time- and resource-intensive…

Deep Reinforcement Learning GPU reinforcement-learning Reinforcement Learning+1

A Reinforcement Learning Environment for Automatic Code Optimization in the MLIR Compiler

2024-09-17 · Nazim Bendib, Iheb Nassim Aouadj, Riyadh Baghdadi

Code optimization is a crucial task aimed at enhancing code performance. However, this process is often tedious and complex, highlighting the necessity for automatic code optimization techniques. Reinforcement Learning (…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)