# Group Sequence Policy Optimization

**Authors:** Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, Junyang Lin  
**arXiv:** [2507.18071](https://arxiv.org/abs/2507.18071)  
**Date:** 2025-07-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2507.18071v1/

## Key points

- Group Sequence Policy Optimization (GSPO) is a novel reinforcement learning algorithm designed for training large language models.
- GSPO leverages sequence likelihood to define the importance ratio and implements sequence-level clipping, rewarding, and optimization strategies.
- GSPO outperforms the GRPO algorithm in terms of training efficiency and performance.
- GSPO stabilizes Mixture-of-Experts (MoE) RL training processes and simplifies the design of reinforcement learning infrastructure.
- The advancements in GSPO have led to significant improvements in the latest Qwen3 models.
- GSPO represents a stable, efficient, and performant solution for training large language models using reinforcement learning techniques.

## Summary

In their paper titled "Group Sequence Policy Optimization," authors Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu,
Rui Men, An Yang, Jingren Zhou and Junyang Lin introduce Group Sequence Policy Optimization (GSPO), a novel reinforcement learning algorithm designed for training large language models. Unlike traditional approaches that rely on token-level importance ratios,
GSPO leverages sequence likelihood to define the importance ratio and implements sequence-level clipping,
rewarding and optimization strategies. The authors demonstrate that GSPO outperforms the GRPO algorithm in terms of training efficiency and performance. Notably, GSPO is shown to stabilize Mixture-of-Experts (MoE) RL training processes and has the potential to simplify the design of reinforcement learning infrastructure. These advancements have led to significant improvements in the latest Qwen3 models. Overall,
GSPO represents a stable, efficient and performant solution for training large language models using reinforcement learning techniques. The algorithm's emphasis on sequence-level considerations sets it apart from existing methods and showcases its potential for driving advancements in natural language processing research.