# A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications

**Authors:** Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, Aman Chadha  
**arXiv:** [2402.07927](https://arxiv.org/abs/2402.07927)  
**Date:** 2024-02-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.07927v1/

## Key points

- Prompt engineering is crucial for enhancing large language models (LLMs) and vision-language models (VLMs) by leveraging task-specific instructions known as prompts.
- Prompts enable model efficacy to be enhanced without modifying core model parameters, facilitating seamless integration into various downstream tasks.
- The field of prompt engineering has advanced significantly, focusing on improving model performance in applications like question-answering and commonsense reasoning.
- Comprehensive reasoning preparation involves establishing context, framing the problem, and gathering evidence from internal knowledge bases, external databases, and given prompts to guide effective model behavior.
- Active Prompting uses task-specific example prompts with chain-of-thought reasoning to enhance LLM performance on complex question-and-answer tasks.
- Fine-tuning techniques like Automatic Prompt Engineer (APE) dynamically generate impactful prompts for specific tasks using reinforcement learning, outperforming human-authored prompts in most cases.
- Knowledge-based reasoning techniques like Automatic Reasoning and Tool-use (ART) empower LLMs to tackle complex problems by integrating external expertise and tools seamlessly.
- Techniques like Thread of Thought (ThoT) enhance LLMs' reasoning abilities within chaotic contexts by systematically examining extensive information into manageable segments for incremental analysis.

## Summary

Prompt engineering has become a crucial technique for enhancing the capabilities of large language models (LLMs) and vision-language models (VLMs) by leveraging task-specific instructions known as prompts. These prompts enable model efficacy to be enhanced without the need to modify core model parameters, allowing for seamless integration of pre-trained models into various downstream tasks. The field of prompt engineering has seen significant advancements in recent years, with a focus on improving model performance across a wide range of applications, from question-answering to commonsense reasoning. One key aspect of prompt engineering is the comprehensive reasoning preparation stage, where context is established and the problem is framed before engaging in dynamic knowledge adaptation. This involves gathering evidence from internal knowledge bases, external databases, and the given prompt to guide the model's behavior effectively. In terms of user interface, Active Prompting has been introduced as a solution to adapting LLMs to diverse reasoning tasks. This method utilizes task-specific example prompts with chain-of-thought (CoT) reasoning to enhance performance on complex question-and-answer tasks. Active Prompting outperforms existing methods by determining impactful questions for annotation through uncertainty-based active learning metrics. Fine-tuning and optimization techniques such as Automatic Prompt Engineer (APE) have revolutionized prompt generation for LLMs by dynamically generating and selecting impactful prompts for specific tasks using reinforcement learning. APE has shown superior performance compared to human-authored prompts in most cases, boosting LLMs' reasoning abilities significantly. Knowledge-based reasoning and generation techniques like Automatic Reasoning and Tool-use (ART) have been developed to empower LLMs in tackling complex problems by seamlessly integrating external expertise and tools. ART enables LLMs to reason through multi-step processes and expand beyond simple text generation, unlocking unprecedented versatility across diverse fields. Moreover, prompting techniques like Thread of Thought (ThoT) have been designed to enhance LLMs' reasoning abilities within chaotic contexts by systematically examining extensive information into manageable segments for incremental analysis. ThoT's flexibility allows it to be easily integrated into different models and prompting methods, enhancing reasoning capabilities across various applications. Overall, these advancements in prompt engineering have paved the way for LLMs and VLMs to tackle a wider range of tasks with greater efficiency and adaptability while unlocking their full potential across diverse applications.