# It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners

**Authors:** Timo Schick, Hinrich Schütze  
**arXiv:** [2009.07118](https://arxiv.org/abs/2009.07118)  
**Date:** 2020-09-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2009.07118v2/

## Key points

- Authors Timo Schick and Hinrich Schütze explore the potential of smaller language models in achieving remarkable few-shot performance similar to that of large pretrained models like GPT-3.
- The study highlights the innovation of converting textual inputs into cloze questions containing task descriptions, which are then optimized using gradient-based techniques.
- This approach not only leads to impressive results but also addresses concerns around environmental impact and accessibility challenges associated with large-scale models.
- By leveraging unlabeled data and identifying crucial factors for successful natural language understanding with small language models, the authors pave the way for more sustainable and accessible approaches to advanced language processing tasks.
- Accepted at NAACL 2021, this research sheds light on the potential of compact language models to achieve high levels of performance while minimizing environmental impact and computational requirements.
- Through their innovative approach and insightful findings, Schick and Schütze contribute valuable insights to the field of natural language processing, offering a promising alternative to resource-intensive large-scale models.

## Summary

In their paper titled "It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners," authors Timo Schick and Hinrich Schütze explore the potential of smaller language models in achieving remarkable few-shot performance similar to that of large pretrained models like GPT-3. The study highlights the key innovation of converting textual inputs into cloze questions containing task descriptions, which are then optimized using gradient-based techniques. This approach not only leads to impressive results but also addresses concerns around the environmental impact and accessibility challenges associated with large-scale models. By leveraging unlabeled data and identifying crucial factors for successful natural language understanding with small language models, the authors pave the way for more sustainable and accessible approaches to advanced language processing tasks. Accepted at NAACL 2021, this research sheds light on the potential of compact language models to achieve high levels of performance while minimizing environmental impact and computational requirements. Through their innovative approach and insightful findings, Schick and Schütze contribute valuable insights to the field of natural language processing, offering a promising alternative to resource-intensive large-scale models.