Prompting Is Programming: A Query Language For Large Language Models
Auteurs : Luca Beurer-Kellner, Marc Fischer, Martin Vechev
Résumé scientifique
Large language models have demonstrated outstanding performance on a wide range of tasks such as question answering and code generation. On a high level, given an input, a language model can be used to automatically complete the sequence in a statistically-likely way. Based on this, users prompt these models with language instructions or examples, to implement a variety of downstream tasks. Advanced prompting methods can even imply interaction between the language model, a user, and external tools such as calculators. However, to obtain state-of-the-art performance or adapt language models for specific tasks, complex task- and model-specific programs have to be implemented, which may still require ad-hoc interaction. Based on this, we present the novel idea of Language Model Programming (LMP). LMP generalizes language model prompting from pure text prompts to an intuitive combination of text prompting and scripting. Additionally, LMP allows constraints to be specified over the language model output. This enables easy adaption to many tasks, while abstracting language model internals and providing high-level semantics. To enable LMP, we implement LMQL (short for Language Model Query Language), which leverages the constraints and control flow from an LMP prompt to generate an efficient inference procedure that minimizes the number of expensive calls to the underlying language model. We show that LMQL can capture a wide range of state-of-the-art prompting methods in an intuitive way, especially facilitating interactive flows that are challenging to implement with existing high-level APIs. Our evaluation shows that we retain or increase the accuracy on several downstream tasks, while also significantly reducing the required amount of computation or cost in the case of pay-to-use APIs (13-85% cost savings).
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
Automatic Prompt Selection for Large Language ModelsViet-Tung Do, Van-Khanh Hoang, Duy-Hung Nguyen et al. · 2024 · Computation and Language · preprint, non relu par les pairs · cité 16 fois · Résumé IA disponible
Résumé scientifique
Large Language Models (LLMs) can perform various natural language processing tasks with suitable instruction prompts. However, designing effective prompts manually is challenging and time-consuming. Existing methods for automatic prompt optimization either lack flexibility or efficiency. In this paper, we propose an effective approach to automatically select the optimal prompt for a given input from a finite set of synthetic candidate prompts. Our approach consists of three steps: (1) clustering the training data and generating candidate prompts for each cluster using an LLM-based prompt generator; (2) synthesizing a dataset of input-prompt-output tuples for training a prompt evaluator to ra…
-
Large Language Models Are Human-Level Prompt EngineersYongchao Zhou, Andrei Ioan Muresanu, Ziwen Han et al. · 2022 · Machine Learning · cité 1 708 fois · Résumé IA disponible
Résumé scientifique
By conditioning on natural language instructions, large language models (LLMs) have displayed impressive capabilities as general-purpose computers. However, task performance depends significantly on the quality of the prompt used to steer the model, and most effective prompts have been handcrafted by humans. Inspired by classical program synthesis and the human approach to prompt engineering, we propose Automatic Prompt Engineer (APE) for automatic instruction generation and selection. In our method, we treat the instruction as the "program," optimized by searching over a pool of instruction candidates proposed by an LLM in order to maximize a chosen score function. To evaluate the quality o…
-
Prompt Engineering and Optimization in Large Language ModelsRanjith Gundam, Kalyan Mandadhi · 2026 · International Scientific Journal of Engineering and Management · Pas encore résumé
Résumé scientifique
Abstract: Large Language Models (LLMs) have emerged as transformative technologies, enabling sophisticated natural language understanding and generation across diverse applications. However, their effectiveness is heavily dependent on how instructions and queries are formulated. Prompt engineering—the art and science of crafting optimal input instructions—has become critical to unlocking the full potential of these models. This systematic review investigates contemporary approaches to prompt engineering and optimization in LLMs, examining techniques such as few-shot learning, chain-of-thought prompting, retrieval augmented generation, and dynamic prompt optimization. The study analyzes vario…
-
Reflective Prompt Tuning through Language Model Function-CallingFarima Fatahi Bayat, Moin Aminnaseri, Pouya Pezeshkpour et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 2 fois · Pas encore résumé
Résumé scientifique
Large language models (LLMs) have become increasingly capable of following instructions and complex reasoning, making prompting a flexible interface for adapting models without parameter updates. Yet prompt design remains labor-intensive and highly sensitive to formatting, phrasing, and instruction order, motivating automated prompt optimization methods that reduce manual effort while preserving inference-time flexibility. However, existing methods often search over prompt candidates or use fixed critique-refine pipelines driven by individual examples or small batches, limiting their ability to capture systematic error patterns and make targeted edits grounded in failure history. We propose …
-
Imprompt: A Language Framework for Prompt ProgrammingChentian Wu, Shengyuan Yang, Adithya Murali · 2026 · arXiv.org · preprint, non relu par les pairs · Pas encore résumé
Résumé scientifique
With the unprecedented success of Language Models (LMs), the science of Prompt Engineering has evolved the powerful idea of Prompt Programming, where prompts are treated as a programmable control surface for describing complex tasks and leveraging LM capabilities. However, existing prompt programming frameworks suffer from various complexities and inelegances, which make them hard to utilize in practice for effectively describing tasks. We propose Imprompt, a new language framework for the study and practice of prompt programming. We undertake a foundational investigation of prompt programming, and contend that prompt programs must contain only the task descriptions and must be decoupled fro…
-
Enhancing LLM Efficiency: A Literature Review of Emerging Prompt Optimization StrategiesAsyafa Ditra Al Hauna, Andi Prademon Yunus, Masanori Fukui et al. · 2025 · International Journal on Robotics Automation and Sciences · cité 3 fois · Pas encore résumé
Résumé scientifique
This study focuses on enhancing the performance of Large Language Models (LLMs) through innovative prompt engineering techniques aimed at optimizing outputs without the high computational costs of model fine-tuning or retraining. The primary objective is to investigate efficient alternatives, such as black-box prompt optimization and ontology-based prompt refinement, which improve LLM performance by refining prompts externally while maintaining the model's internal parameters. The study explores various prompt optimization techniques, including instruction-based, role-based, question-answering, and contextual prompting, alongside advanced methods like CoT and ToT prompting. Methodologically,…
-
Progressive-Hint Prompting Improves Reasoning in Large Language ModelsChuanyang Zheng, Zheng-Ying Liu, En-Ze Xie et al. · 2023 · arXiv.org · preprint, non relu par les pairs · cité 165 fois · Pas encore résumé
Résumé scientifique
The performance of Large Language Models (LLMs) in reasoning tasks depends heavily on prompt design, with Chain-of-Thought (CoT) and self-consistency being critical methods that enhance this ability. However, these methods do not fully exploit the answers generated by the LLM to guide subsequent responses. This paper proposes a new prompting method, named Progressive-Hint Prompting (PHP), that enables automatic multiple interactions between users and LLMs by using previously generated answers as hints to progressively guide toward the correct answers. PHP is orthogonal to CoT and self-consistency, making it easy to combine with state-of-the-art techniques to further improve performance. We c…
-
Exploring Techniques and Applications of Prompt Engineering for Large Language ModelsPrincy Medona, L. A. Felicien, N. Mani · 2025 · International Journal of Scientific Research in Engineering and Management · Pas encore résumé
Résumé scientifique
Abstract - Prompt engineering has become an effective approach for improving the performance of large language models (LLMs) and vision–language models (VLMs) without modifying their internal architecture. By designing precise instructions, structured formats, and illustrative examples, prompts guide models to interpret tasks more accurately and generate more reliable outputs. Despite its rapid growth and significant impact on areas such as question answering, reasoning, and multimodal tasks, the field still lacks a well-organized overview of its diverse techniques. This paper presents a structured survey of recent advances in prompt engineering, explaining how each technique functions, the …
Les travaux antérieurs sur lesquels il s’appuie
-
Prompt Programming for Large Language Models: Beyond the Few-Shot ParadigmLaria Reynolds, Kyle McDonell · 2021 · Computation and Language · cité 1 462 fois · Résumé IA disponible
Résumé scientifique
Prevailing methods for mapping large generative language models to supervised tasks may fail to sufficiently probe models' novel capabilities. Using GPT-3 as a case study, we show that 0-shot prompts can significantly outperform few-shot prompts. We suggest that the function of few-shot examples in these cases is better described as locating an already learned task rather than meta-learning. This analysis motivates rethinking the role of prompts in controlling and evaluating powerful language models. In this work, we discuss methods of prompt programming, emphasizing the usefulness of considering prompts through the lens of natural language. We explore techniques for exploiting the capacity …
-
Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans et al. · 2022 · Computation and Language · cité 23 723 fois · Résumé IA disponible
Résumé scientifique
We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking. For instance, prompting a 540B-parameter language model with just…
-
PAL: Program-aided Language ModelsLuyu Gao, Aman Madaan, Shuyan Zhou et al. · 2022 · Computation and Language · cité 910 fois · Résumé IA disponible
Résumé scientifique
Large language models (LLMs) have recently demonstrated an impressive ability to perform arithmetic and symbolic reasoning tasks when provided with a few examples at test time (few-shot prompting). Much of this success can be attributed to prompting methods for reasoning, such as chain-of-thought, that employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the problem. While LLMs seem to be adept at this sort of step-by-step decomposition, LLMs often make logical and arithmetic mistakes in the solution part, even when the problem is correctly decomposed. We present Program-Aided Language models (PaL): a new method that uses th…
-
ReAct: Synergizing Reasoning and Acting in Language ModelsShunyu Yao, Jeffrey Zhao, Dian Yu et al. · 2022 · International Conference on Learning Representations · cité 13 626 fois · Pas encore résumé
Résumé scientifique
While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics. In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional in…
-
Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder et al. · 2020 · Neural Information Processing Systems · cité 66 241 fois · Résumé IA disponible
Résumé scientifique
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Spec…
Les articles qui le citent
-
DSPy: Compiling Declarative Language Model Calls into Self-Improving PipelinesOmar Khattab, Arnav Singhvi, Paridhi Maheshwari et al. · 2023 · Computation and Language · preprint, non relu par les pairs · cité 1 163 fois · Résumé IA disponible
Résumé scientifique
The ML community is rapidly exploring techniques for prompting language models (LMs) and for stacking them into pipelines that solve complex tasks. Unfortunately, existing LM pipelines are typically implemented using hard-coded "prompt templates", i.e. lengthy strings discovered via trial and error. Toward a more systematic approach for developing and optimizing LM pipelines, we introduce DSPy, a programming model that abstracts LM pipelines as text transformation graphs, i.e. imperative computational graphs where LMs are invoked through declarative modules. DSPy modules are parameterized, meaning they can learn (by creating and collecting demonstrations) how to apply compositions of prompti…
-
ART: Automatic multi-step reasoning and tool-use for large language modelsBhargavi Paranjape, Scott Lundberg, Sameer Singh et al. · 2023 · Computation and Language · preprint, non relu par les pairs · cité 239 fois · Résumé IA disponible
Résumé scientifique
Large language models (LLMs) can perform complex reasoning in few- and zero-shot settings by generating intermediate chain of thought (CoT) reasoning steps. Further, each reasoning step can rely on external tools to support computation beyond the core LLM capabilities (e.g. search/running code). Prior work on CoT prompting and tool use typically requires hand-crafting task-specific demonstrations and carefully scripted interleaving of model generations with tool use. We introduce Automatic Reasoning and Tool-use (ART), a framework that uses frozen LLMs to automatically generate intermediate reasoning steps as a program. Given a new task to solve, ART selects demonstrations of multi-step reas…
-
SGLang: Efficient Execution of Structured Language Model ProgramsLian-Min Zheng, Liangsheng Yin, Zhi-Qiang Xie et al. · 2023 · Neural Information Processing Systems · cité 1 843 fois · Résumé IA disponible
Résumé scientifique
Large language models (LLMs) are increasingly used for complex tasks that require multiple generation calls, advanced prompting techniques, control flow, and structured inputs/outputs. However, efficient systems are lacking for programming and executing these applications. We introduce SGLang, a system for efficient execution of complex language model programs. SGLang consists of a frontend language and a runtime. The frontend simplifies programming with primitives for generation and parallelism control. The runtime accelerates execution with novel optimizations like RadixAttention for KV cache reuse and compressed finite state machines for faster structured output decoding. Experiments show…
-
APPL: A Prompt Programming Language for Harmonious Integration of Programs and Large Language Model PromptsHonghua Dong, Qidong Su, Yu-Bo Gao et al. · 2024 · Annual Meeting of the Association for Computational Linguistics · cité 12 fois · Pas encore résumé
Résumé scientifique
Large Language Models (LLMs) have become increasingly capable of handling diverse tasks with the aid of well-crafted prompts and integration of external tools, but as task complexity rises, the workflow involving LLMs can be complicated and thus challenging to implement and maintain. To address this challenge, we propose APPL, A Prompt Programming Language that acts as a bridge between computer programs and LLMs, allowing seamless embedding of prompts into Python functions, and vice versa. APPL provides an intuitive and Python-native syntax, an efficient parallelized runtime with asynchronous semantics, and a tracing module supporting effective failure diagnosis and replaying without extra c…
-
AskIt: Unified Programming Interface for Programming with Large Language ModelsKatsumi Okuda, Saman P. Amarasinghe · 2023 · IEEE/ACM International Symposium on Code Generation and Optimization · cité 10 fois · Pas encore résumé
Résumé scientifique
Large Language Models (LLMs) exhibit a unique phenomenon known as emergent abilities, demonstrating adeptness across numerous tasks, from text summarization to code generation. While these abilities open up novel avenues in software design and crafting, their incorporation presents substantial challenges. Developers face decisions regarding the use of LLMs for directly performing tasks within applications as well as for generating and executing code to accomplish these tasks. Moreover, effective prompt design becomes a critical concern, given the necessity of extracting data from natural language outputs. To address these complexities, this paper introduces AskIt, a domain-specific language …
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 6 novembre 2026.