# Executable Code Actions Elicit Better LLM Agents

**Authors:** Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, Heng Ji  
**arXiv:** [2402.01030](https://arxiv.org/abs/2402.01030)  
**Date:** 2024-02-01  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.01030v1/

## Key points

- The study explores the potential of Large Language Model (LLM) agents in addressing real-world challenges by leveraging executable Python code to enhance their actions.
- LLM agents are capable of a wide range of tasks but are often limited by constrained action spaces and restricted flexibility.
- The proposed approach, CodeAct, consolidates LLM agents' actions into a unified action space by integrating with a Python interpreter to execute code actions dynamically.
- Extensive analysis involving 17 LLMs on API-Bank and a newly curated benchmark demonstrates that CodeAct outperforms existing alternatives by up to 20% in terms of success rate.
- A dataset called CodeActInstruct consisting of 7k multi-turn interactions using CodeAct has been collected to facilitate interaction between users and the open-source LLM agent developed through this study.
- CodeAct can be utilized with existing data to enhance models in agent-oriented tasks without compromising their overall capability.
- The CodeActAgent, fine-tuned from Llama2 and Mistral models, is integrated with a Python interpreter and specialized for performing complex tasks like model training using existing libraries while autonomously self-debugging.
- Experiments conducted with various LLMs validate the benefits of CodeAct in improving agent performance on basic tasks involving atomic tool use.

## Summary

The study "Executable Code Actions Elicit Better LLM Agents" explores the potential of Large Language Model (LLM) agents in addressing real-world challenges by leveraging executable Python code to enhance their actions. These LLM agents are capable of a wide range of tasks, such as invoking tools and controlling robots, but are often limited by constrained action spaces and restricted flexibility. The proposed approach, CodeAct, consolidates LLM agents' actions into a unified action space by integrating with a Python interpreter to execute code actions dynamically. Extensive analysis involving 17 LLMs on API-Bank and a newly curated benchmark demonstrates that CodeAct outperforms existing alternatives by up to 20% in terms of success rate. This performance improvement has led to the development of an open-source LLM agent that interacts with environments through interpretable code execution and collaborates with users using natural language. To facilitate this interaction, the researchers have collected a dataset called CodeActInstruct consisting of 7k multi-turn interactions using CodeAct. Furthermore, the study showcases how CodeAct can be utilized with existing data to enhance models in agent-oriented tasks without compromising their overall capability. The CodeActAgent, fine-tuned from Llama2 and Mistral models, is integrated with a Python interpreter and specialized for performing complex tasks like model training using existing libraries while autonomously self-debugging. The experiments conducted with various LLMs validate the benefits of CodeAct in improving agent performance on basic tasks involving atomic tool use. By enabling more flexible and dynamic actions through executable code, CodeAct offers promising advancements in enhancing the capabilities of LLM agents for tackling diverse real-world challenges effectively.