# How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition

**Authors:** Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, Jingren Zhou  
**arXiv:** [2310.05492](https://arxiv.org/abs/2310.05492)  
**Date:** 2023-10-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.05492v3/

## Key points

- Study focuses on impact of supervised fine-tuning (SFT) data composition on large language models (LLMs)
- LLMs have diverse capabilities: math reasoning, code generation, instruction following
- Open-source LLMs enhanced through ad-hoc SFT, proprietary LLMs versatile across skills
- Four research questions proposed to explore association between model performance and factors like data amount, composition ratio, model size, SFT strategies
- Different capabilities of LLMs scale differently, larger models generally show superior performance with same amount of data
- Mathematical reasoning and code generation consistently improve with increasing data amount
- General human-aligning abilities plateau after approximately a thousand samples
- Data composition can enhance various abilities under limited data conditions but may lead to performance conflicts with abundance of data
- Amount of composition data has greater influence on performance than composition ratio
- Sequentially learning multiple skills risks catastrophic forgetting
- Dual-stage Mixed Fine-tuning (DMT) strategy proposed as solution for learning multiple abilities with different scaling patterns

## Summary

, , , , 

This study focuses on the impact of supervised fine-tuning (SFT) data composition on the abilities of large language models (LLMs). LLMs have diverse capabilities, such as math reasoning, code generation, and instruction following. Open-source LLMs have been enhanced through ad-hoc SFT, while proprietary LLMs exhibit versatility across various skills. Understanding how SFT facilitates multiple abilities is crucial for improving model performance. The researchers propose four research questions to explore the association between model performance and factors like data amount, composition ratio, model size, and SFT strategies. The experiments reveal that different capabilities of LLMs scale differently, with larger models generally showing superior performance with the same amount of data. Mathematical reasoning and code generation consistently improve with increasing data amount, while general human-aligning abilities plateau after approximately a thousand samples. Data composition can enhance various abilities under limited data conditions but may lead to performance conflicts when there is an abundance of data. Additionally, the amount of composition data has a greater influence on performance than the composition ratio. In analyzing SFT strategies, it is discovered that sequentially learning multiple skills risks catastrophic forgetting. To address this issue, the researchers propose a Dual-stage Mixed Fine-tuning (DMT) strategy that offers a promising solution for learning multiple abilities with different scaling patterns. Overall, this study provides insights into how SFT data composition affects the abilities of large language models and proposes strategies to optimize their performance in various tasks.