Reinforcement Pre-Training

2025-06-09 · Qingxiu Dong, Li Dong, Yao Tang, Tianzhu Ye, Yutao Sun, Zhifang Sui, Furu Wei

In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling paradigm for large language models and reinforcement learning (RL). Specifically, we reframe next-token prediction as a reasoning task trained using RL, where it receives verifiable rewards for correctly predicting the next token for a given context. RPT offers a scalable method to leverage vast amounts of text data for general-purpose RL, rather than relying on domain-specific annotated answers. By incentivizing the capability of next-token reasoning, RPT significantly improves the language modeling accuracy of predicting the next tokens. Moreover, RPT provides a strong pre-trained foundation for further reinforcement fine-tuning. The scaling curves show that increased training compute consistently improves the next-token prediction accuracy. The results position RPT as an effective and promising scaling paradigm to advance language model pre-training.

📄 PDF Abstract BibTeX arXiv:2506.08007

Code (0)

등록된 구현이 없습니다.

Tasks

Language ModelingLanguage Modelling Reinforcement Learning (RL)

Non-Robust Feature Mapping in Deep Reinforcement Learning

2021-06-18 · ICML Workshop AML 2021 7 · Ezgi Korkmaz

Adversarial perturbations to state observations can dramatically degrade the performance of deep reinforcement learning policies, and thus raise concerns regarding the robustness of deep reinforcement learning agents. A …

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Reinforcement Networks: novel framework for collaborative Multi-Agent Reinforcement Learning tasks

2025-12-28 · Maksim Kryzhanovskiy, Svetlana Glazyrina, Roman Ischenko, Konstantin Vorontsov arxiv

Modern AI systems often comprise multiple learnable components that can be naturally organized as graphs. A central challenge is the end-to-end training of such systems without restrictive architectural or training assum…

Multi-agent Reinforcement Learning

Pseudo-Model-Free Hedging for Variable Annuities via Deep Reinforcement Learning

2021-07-07 · Wing Fung Chong, Haoen Cui, YuXuan Li

This paper proposes a two-phase deep reinforcement learning approach, for hedging variable annuity contracts with both GMMB and GMDB riders, which can address model miscalibration in Black-Scholes financial and constant …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Benchmarking Robustness of Deep Reinforcement Learning approaches to Online Portfolio Management

2023-06-19 · Marc Velay, Bich-Liên Doan, Arpad Rimmel, Fabrice Popineau 외

Deep Reinforcement Learning approaches to Online Portfolio Selection have grown in popularity in recent years. The sensitive nature of training Reinforcement Learning agents implies a need for extensive efforts in market…

Benchmarking Deep Reinforcement Learning Management reinforcement-learning+1

QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learning

2019-10-02 · Srivatsan Krishnan, Maximilian Lam, Sharad Chitlangia, Zishen Wan 외

Deep reinforcement learning continues to show tremendous potential in achieving task-level autonomy, however, its computational and energy demands remain prohibitively high. In this paper, we tackle this problem by apply…

Decision Making Deep Reinforcement Learning Quantization reinforcement-learning+2