# DeLLMa: Decision Making Under Uncertainty with Large Language Models

**Authors:** Ollie Liu, Deqing Fu, Dani Yogatama, Willie Neiswanger  
**arXiv:** [2402.02392](https://arxiv.org/abs/2402.02392)  
**Date:** 2024-02-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.02392v3/

## Key points

- Authors explore potential of large language models (LLMs) as decision support tools in various fields
- Directly prompting LLMs on decision-making problems may not yield accurate results, especially with increasing problem complexity
- Introduction of DeLLMa (Decision-making Large Language Model assistant) to improve decision-making accuracy in uncertain environments
- DeLLMa incorporates multi-step reasoning procedure, scaling inference-time reasoning best practices, and principles from decision theory and utility theory
- Validation shows DeLLMa outperforms leading language models by up to 40% in accuracy
- Performance improves with scaling compute at test time and human evaluations benchmark components of DeLLMa
- Research highlights importance of specialized frameworks like DeLLMa for enhancing decision-making processes using LLMs

## Summary

In their paper "DeLLMa: Decision Making Under Uncertainty with Large Language Models," authors Ollie Liu, Deqing Fu, Dani Yogatama, and Willie Neiswanger from the University of Southern California explore the potential of large language models (LLMs) as decision support tools in various fields. These fields often require decision-making under uncertainty, which can be challenging for traditional methods. The authors emphasize that directly prompting LLMs on decision-making problems may not yield accurate results, especially as problem complexity increases. To address this issue, they introduce DeLLMa (Decision-making Large Language Model assistant), a framework designed to improve decision-making accuracy in uncertain environments. DeLLMa incorporates a multi-step reasoning procedure that integrates best practices in scaling inference-time reasoning and draws upon principles from decision theory and utility theory to provide an accurate and human-auditable decision-making process. Through validation on multiple realistic decision-making environments, the authors demonstrate that DeLLMa consistently outperforms leading language models by up to 40% in terms of accuracy. They also show how performance further improves when scaling compute at test time and conduct human evaluations to benchmark components of DeLLMa. Overall, this research highlights the importance of specialized frameworks like DeLLMa in enhancing decision-making processes in uncertain environments using large language models. It offers valuable insights for industries facing complex decision-making challenges and paves the way for more effective utilization of LLMs as decision support tools across various domains.