# LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement

**Authors:** Nicholas Lee, Thanakul Wattanawong, Sehoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipali, Michael W. Mahoney, Kurt Keutzer, Amir Gholami  
**arXiv:** [2403.15042](https://arxiv.org/abs/2403.15042)  
**Date:** 2024-03-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.15042v1/

## Key points

- Pretrained large language models (LLMs) are cutting-edge solutions for natural language processing tasks
- Real-world applications often require fine-tuning for optimal performance, especially with limited data availability
- LLM2LLM is a novel approach that leverages a teacher LLM to augment data for fine-tuning on specific tasks
- The process involves fine-tuning a student LLM, identifying incorrect data points, generating synthetic data with the teacher LLM, and incorporating it back into training
- LLM2LLM enhances performance in low-data scenarios by focusing on challenging examples and amplifying signals from incorrectly predicted instances
- Results show superiority over traditional fine-tuning methods and other data augmentation techniques, with significant improvements observed across various datasets

## Summary

Pretrained large language models (LLMs) have emerged as the cutting-edge solution for a wide array of tasks in the realm of natural language processing. However, many real-world applications still require fine-tuning to achieve optimal performance levels, especially when data availability is limited. To address this challenge, a novel approach known as LLM2LLM has been introduced. This targeted and iterative data augmentation strategy leverages a teacher LLM to enrich a small seed dataset by generating additional data for fine-tuning on specific tasks. The process begins with fine-tuning a baseline student LLM on the initial seed data. The model is then evaluated to identify and extract incorrect data points where it makes errors. These incorrect data points are used by the teacher LLM to generate synthetic data, which is incorporated back into the training set. By amplifying the signal from incorrectly predicted instances during training and focusing on more challenging examples, LLM2LLM significantly enhances the performance of LLMs in low-data scenarios. The results obtained from this innovative approach showcase its superiority over traditional fine-tuning methods and other data augmentation techniques. By reducing reliance on labor-intensive data curation, LLM2LLM opens up avenues for more scalable and efficient LLM solutions, enabling researchers to effectively address challenges posed by data-constrained domains and tasks. Significant improvements have been observed across various datasets using this approach, including a remarkable enhancement of up to 24.2% on the GSM8K dataset, 32.6% on CaseHOLD, 32.0% on SNIPS, 52.6% on TREC, and 39.8% on SST-2 compared to regular fine-tuning approaches in low-data settings using an LLaMA2-7B student model.