# Self-Alignment with Instruction Backtranslation

**Authors:** Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, Mike Lewis  
**arXiv:** [2308.06259](https://arxiv.org/abs/2308.06259)  
**Date:** 2023-08-11  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2308.06259v2/

## Key points

- Approach involves finetuning a language model on seed data and web corpus
- Seed model used to generate instruction prompts for web documents
- Self-curation of high-quality examples from generated prompts
- Use resulting data to further finetune the model
- Analysis conducted on importance of data quality vs quantity in learning to follow instructions
- Improving data quality significantly improves performance even with smaller dataset sizes
- Prior work suggested only a few thousand high-quality examples were sufficient, but this study found otherwise
- Efficiency of data scaling evaluated by comparing performance of different instruction-following models with varying amounts of finetune data used
- Instruction backtranslation method outperformed other methods using instruction datasets from different sources
- Data quality is important in achieving strong performance, citing previous approaches that curated high-quality human-written data
- Their approach provides a recipe for building a strong model from scratch, without relying on knowledge distillation from other models
- Findings highlight effectiveness of instruction backtranslation approach and emphasize significance of both data quality and quantity in achieving optimal performance.

## Summary

This approach involves finetuning a language model on a small amount of seed data and a web corpus. The seed model is used to generate instruction prompts for web documents which are then self-curated to select high-quality examples. This resulting data is then used to further finetune the model. The researchers conducted an analysis to understand the importance of data quality versus data quantity in learning to follow instructions. They compared finetuning on augmented data of different quality levels and found that improving the quality of training data significantly improves performance even with smaller dataset sizes. This contrasts with prior work which suggested only a few thousand high-quality examples were sufficient for alignment. They also evaluated the efficiency of data scaling by comparing the performance of various instruction-following models as they altered the amount of finetune data used. The win rate was measured against a baseline model and an estimate of efficiency was reported using a scaling coefficient. Their instruction backtranslation method outperformed other methods using instruction datasets created from different sources. The researchers discussed the importance of data quality in achieving strong performance, citing previous approaches that curated high-quality human-written data. They also noted that most finetuned LLaMA models rely on knowledge distillation from other strong models but their approach provides a recipe for building a strong model from scratch. Overall, their findings highlight the effectiveness of their instruction backtranslation approach in building a high-quality instruction-following language model and emphasize the significance of both data quality and quantity in achieving optimal performance.