# An Empirical Study of Instruction-tuning Large Language Models in Chinese

**Authors:** Qingyi Si, Tong Wang, Zheng Lin, Xu Zhang, Yanan Cao, Weiping Wang  
**arXiv:** [2310.07328](https://arxiv.org/abs/2310.07328)  
**Date:** 2023-10-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.07328v2/

## Key points

- Authors Qingyi Si, Tong Wang, Zheng Lin, Xu Zhang, Yanan Cao, and Weiping Wang conduct an empirical study on instruction-tuning Large Language Models (LLMs) in Chinese.
- The study aims to provide insights for customizing LLMs to effectively respond to Chinese instructions.
- Key elements explored include LLM bases, parameter-efficient methods, and instruction data types crucial for instruction-tuning.
- Experiments analyze the impact of factors like chain-of-thought data and human-value alignment on instruction-tuning.
- Findings are expected to contribute significantly to developing an open Chinese version of ChatGPT and advancing large language models for diverse linguistic contexts.

## Summary

, , , , 

In their paper titled "An Empirical Study of Instruction-tuning Large Language Models in Chinese," authors Qingyi Si, Tong Wang, Zheng Lin, Xu Zhang, Yanan Cao, and Weiping Wang delve into the realm of large language models (LLMs) and their potential in artificial general intelligence (AGI). The success of ChatGPT has highlighted the significance of LLMs, prompting interest in instruction-tuning within the open-source community to enhance ChatGPT's replication process. Despite this progress, research on instruction-tuning LLMs in Chinese, the most widely spoken language globally, is still nascent. To address this gap, the authors conduct an extensive empirical study focusing on instruction-tuning LLMs in Chinese. Their work aims to provide valuable insights for customizing LLMs to effectively respond to Chinese instructions. The study systematically explores key elements such as LLM bases, parameter-efficient methods, and instruction data types crucial for instruction-tuning. Additionally, experiments are conducted to analyze the impact of other factors like chain-of-thought data and human-value alignment. The findings from this empirical study are expected to contribute significantly to the development of an open Chinese version of ChatGPT by introducing a powerful Chinese LLM that rivals ChatGLM. The researchers make their code and data available at https://github.com/PhoebusSi/Alpaca-CoT for further exploration and replication. This comprehensive investigation not only sheds light on optimizing LLMs for Chinese but also sets a foundation for future advancements in leveraging large language models for diverse linguistic contexts.