# Automatic Prompt Optimization with "Gradient Descent" and Beam Search

**Authors:** Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, Michael Zeng  
**arXiv:** [2305.03495](https://arxiv.org/abs/2305.03495)  
**Date:** 2023-05-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.03495v1/

## Key points

- Large Language Models (LLMs) are highly dependent on hand-written prompts that require significant trial-and-error effort
- Automatic Prompt Optimization (APO) is a proposed solution to this problem
- APO is a simple and nonparametric algorithm inspired by numerical gradient descent that automatically improves prompts assuming access to training data and an LLM API
- APO uses minibatches of data to form natural language gradients that criticize the current prompt, which are then propagated into the prompt by editing it in the opposite semantic direction of the gradient
- The algorithm's gradient descent steps are guided by a beam search and bandit selection procedure which significantly increases its efficiency
- Preliminary results across three benchmark NLP tasks and the novel problem of LLM jailbreak detection show that APO can outperform prior prompt editing techniques and improve an initial prompt's performance by up to 31% by using data to rewrite vague task descriptions into more precise annotation instructions
- This paper presents a promising approach for improving LLMs' abilities without requiring extensive human input or domain knowledge.

## Summary

In their paper titled "Automatic Prompt Optimization with 'Gradient Descent' and Beam Search," authors Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng propose a solution to the problem of Large Language Models (LLMs) being highly dependent on hand-written prompts that require significant trial-and-error effort. The proposed solution is Automatic Prompt Optimization (APO), a simple and nonparametric algorithm inspired by numerical gradient descent that automatically improves prompts assuming access to training data and an LLM API. APO uses minibatches of data to form natural language gradients that criticize the current prompt. These gradients are then propagated into the prompt by editing it in the opposite semantic direction of the gradient. The algorithm's gradient descent steps are guided by a beam search and bandit selection procedure which significantly increases its efficiency. Preliminary results across three benchmark NLP tasks and the novel problem of LLM jailbreak detection show that APO can outperform prior prompt editing techniques and improve an initial prompt's performance by up to 31% by using data to rewrite vague task descriptions into more precise annotation instructions. This paper presents a promising approach for improving LLMs' abilities without requiring extensive human input or domain knowledge.