# Reinforcement Pre-Training

**Authors:** Qingxiu Dong, Li Dong, Yao Tang, Tianzhu Ye, Yutao Sun, Zhifang Sui, Furu Wei  
**arXiv:** [2506.08007](https://arxiv.org/abs/2506.08007)  
**Date:** 2025-06-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2506.08007v1/

## Key points

- The paper introduces Reinforcement Pre-Training (RPT) as a scaling paradigm for large language models and reinforcement learning (RL)
- RPT reframes next-token prediction as a reasoning task trained using RL techniques
- By incentivizing next-token reasoning capability, RPT significantly enhances accuracy in predicting subsequent tokens
- RPT can leverage vast amounts of text data for general-purpose RL applications without domain-specific annotated answers
- RPT establishes a robust pre-trained foundation that can be further fine-tuned through reinforcement learning methods
- Increased training compute consistently enhances the accuracy of next-token prediction according to scaling curves
- The empirical evidence solidifies RPT as an effective scaling paradigm for language model pre-training methodologies
- RPT enables more efficient and accurate language modeling tasks through reinforcement learning principles

## Summary

The paper "Reinforcement Pre-Training" by Qingxiu Dong, Li Dong, Yao Tang, Tianzhu Ye, Yutao Sun, Zhifang Sui and Furu Wei introduces a groundbreaking scaling paradigm for large language models and reinforcement learning (RL) called Reinforcement Pre-Training (RPT). This approach reframes next-token prediction as a reasoning task trained using RL techniques. By incentivizing the capability of next-token reasoning, RPT significantly enhances the accuracy of language modeling in predicting subsequent tokens. One of its key advantages is its ability to leverage vast amounts of text data for general-purpose RL applications without relying on domain-specific annotated answers. Additionally, RPT establishes a robust pre-trained foundation that can be further fine-tuned through reinforcement learning methods. The authors demonstrate through scaling curves that increased training compute consistently enhances the accuracy of next-token prediction. This empirical evidence solidifies RPT as an effective and promising scaling paradigm for advancing language model pre-training methodologies. Overall, their research showcases how RPT can revolutionize the field by enabling more efficient and accurate language modeling tasks through reinforcement learning principles.