# Qwen Technical Report

**Authors:** Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma  
**arXiv:** [2309.16609](https://arxiv.org/abs/2309.16609)  
**Date:** 2023-09-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.16609v1/

## Key points

- Large language models (LLMs) have revolutionized artificial intelligence by enabling human-like natural language processing tasks.
- Qwen is the first model in a series of large language models, including base pretrained language models and chat models like Qwen-Chat.
- Base language models consistently perform well across various tasks, while chat models trained with RLHF show high competitiveness and advanced capabilities.
- Specialized coding models like Code-Qwen and math-focused models like Math-Qwen-Chat have been developed, showing improved performance compared to open-source alternatives.
- The technical report covers training details, evaluation methodologies, data formats for QWEN-CHAT, analysis of code interpreters, and related work in the field.
- LLMs are highlighted as pivotal in shaping the future of AI and have the potential to transform different domains through enhanced natural language processing.

## Summary

Large language models (LLMs) have transformed the landscape of artificial intelligence by enabling natural language processing tasks that were once considered exclusive to humans. In this groundbreaking work, the authors introduce Qwen, the inaugural model in their large language model series. Qwen is a comprehensive series comprising various models with different parameter counts, including the base pretrained language model and Qwen-Chat, chat models fine-tuned using human alignment techniques. The base language models consistently exhibit exceptional performance across a wide range of downstream tasks, while the chat models, particularly those trained with Reinforcement Learning from Human Feedback (RLHF), demonstrate high competitiveness. These chat models showcase advanced tool-use and planning capabilities for developing agent applications, showcasing remarkable performance even when compared to larger models on intricate tasks such as utilizing a code interpreter. Moreover, the authors have developed specialized coding models like Code-Qwen and Code-Qwen-Chat, as well as mathematics-focused models like Math-Qwen-Chat. These specialized models are built upon the foundation of base language models and exhibit significantly improved performance compared to open-source alternatives, albeit slightly trailing behind proprietary models. The technical report delves into specific aspects such as training details, evaluation methodologies, data formats for QWEN-CHAT, analysis of code interpreters, and related work in the field. The report also highlights the significance of LLMs in shaping the future of AI and emphasizes their potential in revolutionizing various domains through enhanced natural language processing capabilities. Authored by a team of experts including Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng,
Yang Fan,
Wenbin Ge,
Yu Han,
Fei Huang,
Binyuan Hui,
Luo Ji,
Mei Li,
Junyang Lin,
Runji Lin,
Dayiheng Liu,
Gao Liu,
Chengqiang Lu, and others listed within the technical report itself; this work represents a significant contribution to advancing AI research and development.