# Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm

**Authors:** Laria Reynolds, Kyle McDonell  
**arXiv:** [2102.07350](https://arxiv.org/abs/2102.07350)  
**Date:** 2021-02-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2102.07350v1/

## Key points

- The paper explores the theory of prompt programming for large generative language models, with a focus on natural language prompts.
- GPT-3 is used as a case study to demonstrate that 0-shot prompts can outperform few-shot prompts.
- The function of few-shot examples is better described as locating an already learned task rather than meta-learning.
- The authors discuss methods of prompt programming, emphasizing the usefulness of considering prompts through the lens of natural language.
- Techniques for exploiting narratives and cultural anchors to encode nuanced intentions are explored.
- A metaprompt idea is introduced that seeds the model to generate its own natural language prompts for a range of tasks.
- Directions for future work in prompt programming include disentangling meta-learning and task location, developing new benchmarking methods, and creating effective meta-prompt templates for large-scale automated evaluations on closed-ended questions while enabling open-ended reasoning beyond simple fact recall.
- Prompt programming is an emerging field requiring interdisciplinary knowledge and methods with potential to become its own discipline.

## Summary

This paper explores the theory of prompt programming for large generative language models, with a focus on the usefulness of natural language prompts. The authors use GPT-3 as a case study to demonstrate that 0-shot prompts can outperform few-shot prompts, suggesting that the function of few-shot examples is better described as locating an already learned task rather than meta-learning. This analysis motivates rethinking the role of prompts in controlling and evaluating powerful language models. The authors discuss methods of prompt programming, emphasizing the usefulness of considering prompts through the lens of natural language. They explore techniques for exploiting narratives and cultural anchors to encode nuanced intentions and encourage deconstruction of a problem into components before producing a verdict. Informed by this more encompassing theory of prompt programming, they introduce the idea of a metaprompt that seeds the model to generate its own natural language prompts for a range of tasks. The paper also discusses directions for future work in prompt programming, such as disentangling meta-learning and task location; developing new methods for benchmarking like isolating catastrophic failures; and creating effective meta-prompt templates for large-scale automated evaluations on closed-ended questions which still allow some amount of open-ended reasoning. The authors suggest that prompt programming is an emerging field requiring interdisciplinary knowledge and methods with potential to become its own discipline. Overall, this paper provides insights into how natural language prompts can be used effectively to control and evaluate large generative language models. It highlights the need for further research in this area to develop automated methods for prompting and benchmarking these models accurately while enabling open-ended reasoning beyond simple fact recall.