Domain-Adaptable Reinforcement Learning for Code Generation with Dense Rewards

2026-05-20 · Erfan Aghadavoodi Jolfaei, Daniel Maninger, Abhinav Anand, Mert Tiftikci, Mira Mezini arxiv

Large language models show strong potential for automated code generation, but lack guarantees for correctness, quality, safety, and domain-specific constraints. For instance in robotics, where code generation is increasingly being used for planning and executing actions, awareness of the environment and physical constraints is critical. To facilitate the adaption of code-generating LLMs to diverse requirements, including domain-specific ones, we present a reinforcement learning framework that fine-tunes pre-trained LLMs using proximal policy optimization. Our customizable execution-aware reward formula captures and optimizes syntax, functional correctness, code style, security, and simulator executability. A token-level reward mapping mechanism enables effective credit assignment from execution outcomes to generated tokens. The framework is evaluated on general-purpose code generation (MBPP/MBPP+) and robotic program synthesis (RoboEval). The results show substantial improvements in functional correctness and simulator executability, including an absolute pass@1 increase of 19% on MBPP and a reduction in execution failures by 51% on RoboEval. These findings demonstrate that structured reinforcement learning can effectively align language models to correct program generation and domain-specific requirements.

📄 PDF Abstract BibTeX arXiv:2605.21180

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Program Synthesis Code Generation

Efficient Encoder-Decoder and Dual-Path Conformer for Comprehensive Feature Learning in Speech Enhancement

2023-06-09 · Junyu Wang

Current speech enhancement (SE) research has largely neglected channel attention and spatial attention, and encoder-decoder architecture-based networks have not adequately considered how to provide efficient inputs to th…

Decoder Speech Enhancement

Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval

2022-08-11 · Jingtao Zhan, Qingyao Ai, Yiqun Liu, Jiaxin Mao 외

Recent advance in Dense Retrieval (DR) techniques has significantly improved the effectiveness of first-stage retrieval. Trained with large-scale supervised data, DR models can encode queries and documents into a low-dim…

Ad-Hoc Information Retrieval Domain Adaptation Retrieval

Text2Reward: Reward Shaping with Language Models for Reinforcement Learning

2023-09-20 · Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu 외

Designing reward functions is a longstanding challenge in reinforcement learning (RL); it requires specialized knowledge or domain data, leading to high costs for development. To address this, we introduce Text2Reward, a…

MuJoCo reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Trustful LLMs: Customizing and Grounding Text Generation with Knowledge Bases and Dual Decoders

2024-11-12 · Xiaofeng Zhu, Jaya Krishna Mandivarapu

Although people are impressed by the content generation skills of large language models, the use of LLMs, such as ChatGPT, is limited by the domain grounding of the content. The correctness and groundedness of the genera…

Decoder Hallucination RAG Retrieval-augmented Generation+1

Beyond Binary: Turning Partial Success into Dense Verifiable Rewards for Reinforcement Learning in Code Generation

2026-01-07 · Longwen Wang, Yirui Liu, Xuan'er Wu, Xiaohui Hu 외 arxiv

Effective reward design is a central challenge in Reinforcement Learning (RL) for code generation. Mainstream test-suite-level outcome rewards enforce functional correctness but induce sparsity, while external Reward Mod…

Reinforcement Learning Code Generation