# Prompt Engineering a Prompt Engineer

**Authors:** Qinyuan Ye, Maxamed Axmed, Reid Pryzant, Fereshte Khani  
**arXiv:** [2311.05661](https://arxiv.org/abs/2311.05661)  
**Date:** 2023-11-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2311.05661v1/

## Key points

- Authors address the challenge of prompt engineering for optimizing large language models (LLMs)
- Previous works have explored meta-prompting LLMs, but lack guidance for complex reasoning capabilities
- Authors propose "prompt engineering a prompt engineer" to construct a meta-prompt that better guides LLMs
- Key components include step-by-step reasoning template and context specification
- Verbalized optimization concepts are introduced into the meta-prompt
- Final method called PE2 outperforms existing approaches on MultiArith and GSM8K datasets
- PE2 is applied to various scenarios, achieving strong performance in different tasks
- PE2 makes meaningful and targeted prompt edits, addressing errors and exhibiting counterfactual reasoning abilities
- Work contributes to advancing automatic prompt engineering techniques through a refined meta-prompt approach.

## Summary

In the paper titled "Prompt Engineering a Prompt Engineer," authors Qinyuan Ye, Maxamed Axmed, Reid Pryzant, and Fereshte Khani address the challenging task of prompt engineering for optimizing the performance of large language models (LLMs). They highlight the need for complex reasoning to identify errors in LLMs, determine what is missing or misleading in the current prompt, and effectively communicate the task. While previous works have explored meta-prompting LLMs for automatic prompt engineering, their potential remains untapped due to insufficient guidance for eliciting complex reasoning capabilities. To overcome this limitation, the authors propose the problem of "prompt engineering a prompt engineer" and aim to construct a meta-prompt that better guides LLMs in performing automatic prompt engineering. They introduce and analyze key components such as a step-by-step reasoning template and context specification which lead to improved performance. Inspired by optimization concepts like batch size, step size, and momentum they also introduce their verbalized counterparts into the meta-prompt and investigate their effects. The authors present their final method called PE2 which outperforms existing approaches by 6.3% on the MultiArith dataset and 3.1% on the GSM8K dataset. To demonstrate its versatility they apply PE2 to various scenarios including Instruction Induction benchmark counterfactual tasks suite and a real-world industrial prompt where it achieves strong performance surpassing prior automatic prompt engineering baselines. Furthermore they show that PE2 makes meaningful and targeted prompt edits while addressing erroneous or incomplete prompts as well as exhibiting non-trivial counterfactual reasoning abilities. Overall this work contributes to advancing automatic prompt engineering techniques by providing more effective guidance for LLMs through a refined meta-prompt approach.