# Self-Instruct: Aligning Language Models with Self-Generated Instructions

**Authors:** Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, Hannaneh Hajishirzi  
**arXiv:** [2212.10560](https://arxiv.org/abs/2212.10560)  
**Date:** 2022-12-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2212.10560v2/

## Key points

- Large "instruction-tuned" language models rely heavily on limited and repetitive human-written instruction data.
- The proposed framework, Self-Instruct, improves instruction-following capabilities by leveraging the model's own generations.
- Self-Instruct generates instructions, input, and output samples from a language model and filters out invalid or similar ones before fine-tuning the original model.
- When applied to vanilla GPT3, Self-Instruct achieves a 33% absolute improvement on Super-NaturalInstructions compared to InstructGPT-001 trained with private user data and human annotations.
- Expert-written instructions are further curated for novel tasks and human evaluation shows that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin, with only a 5% absolute gap behind InstructGPT-001.
- This annotation-free approach effectively aligns pre-trained language models with instructions.
- A synthetic dataset of 52K instructions and manually written novel tasks is released for future studies on instruction tuning.

## Summary

Large "instruction-tuned" language models have shown impressive generalization abilities to new tasks, but they heavily rely on limited and often repetitive human-written instruction data. To address this limitation, we propose Self-Instruct, a framework that improves instruction-following capabilities by leveraging the model's own generations. Our pipeline generates instructions, input, and output samples from a language model and filters out invalid or similar ones before using them to fine-tune the original model. When applied to vanilla GPT3, our method achieves a 33% absolute improvement on Super-NaturalInstructions, comparable to InstructGPT-001 trained with private user data and human annotations. We further curate expert-written instructions for novel tasks and demonstrate through human evaluation that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin, with only a 5% absolute gap behind InstructGPT-001. This annotation-free approach aligns pre-trained language models with instructions effectively. We release our synthetic dataset of 52K instructions and manually written novel tasks for future studies on instruction tuning.