# LIMA: Less Is More for Alignment

**Authors:** Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, Omer Levy  
**arXiv:** [2305.11206](https://arxiv.org/abs/2305.11206)  
**Date:** 2023-05-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.11206v1/

## Key points

- Large language models are trained in two stages: unsupervised pretraining and large scale instruction tuning and reinforcement learning.
- Almost all knowledge in large language models is learned during pretraining, making limited instruction tuning data necessary for high-quality output.
- LIMA, a 65B parameter LLaMa language model, was trained with only 1,000 carefully curated prompts and responses without any reinforcement learning or human preference modeling.
- LIMA demonstrated strong performance in a controlled human study compared to other models.
- Limitations include the mental effort required for constructing examples and the possibility of weak responses due to unlucky samples or adversarial prompts.
- Scaling up input diversity and output quality has positive effects on alignment while scaling up quantity alone might not.
- Fine-tuning a strong pretrained language model on carefully curated examples can produce remarkable results on a wide range of prompts with limited instruction tuning data necessary for producing high-quality output.

## Summary

Large language models are typically trained in two stages: unsupervised pretraining from raw text to learn general-purpose representations, and large scale instruction tuning and reinforcement learning to better align with end tasks and user preferences. A recent study has shown that almost all knowledge in large language models is learned during pretraining, and only limited instruction tuning data is necessary to teach models to produce high-quality output. The study involved training LIMA, a 65B parameter LLaMa language model, with the standard supervised loss on only 1,000 carefully curated prompts and responses without any reinforcement learning or human preference modeling. LIMA demonstrated remarkably strong performance in a controlled human study; responses from LIMA were either equivalent or strictly preferred to GPT-4 in 43% of cases; this statistic was as high as 58% when compared to Bard and 65% versus DaVinci003 which was trained with human feedback. However, there are limitations associated with this approach such as the mental effort required for constructing examples which is difficult to scale up. In addition, an unlucky sample during decoding or an adversarial prompt can often lead to a weak response. The effects of training data diversity, quality and quantity were also investigated through ablation experiments; it was observed that scaling up input diversity and output quality had measurable positive effects on alignment while scaling up quantity alone might not. Overall, these findings suggest that fine-tuning a strong pretrained language model on carefully curated examples can produce remarkable results on a wide range of prompts with limited instruction tuning data necessary for producing high-quality output. However further research is needed to address some limitations associated with this approach.