# Tree Search for Language Model Agents

**Authors:** Jing Yu Koh, Stephen McAleer, Daniel Fried, Ruslan Salakhutdinov  
**arXiv:** [2407.01476](https://arxiv.org/abs/2407.01476)  
**Date:** 2024-07-01  
**Summary page:** https://www.summarizepaper.com/en/paper/2407.01476v1/

## Key points

- Authors address limitations of autonomous agents powered by language models (LMs) in decision-making tasks
- LMs struggle with multi-step reasoning, planning, and utilizing environmental feedback for realistic computer tasks
- Proposed inference-time search algorithm enables LM agents to conduct exploration and multi-step planning within interactive web environments
- Approach involves implementing a best-first tree search algorithm directly within the environment space
- Demonstrated effectiveness of search algorithm on GPT-4o agent on VisualWebArena benchmark, achieving significant success rate improvements
- Incorporating search algorithm leads to competitive success rates on WebArena as well
- Authors highlight benefits of employing search algorithms for web agents and discuss potential limitations and future research directions
- Code and models developed as part of the study are publicly available at https://jykoh.com/search-agents

## Summary

In their paper titled "Tree Search for Language Model Agents," authors Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov address the limitations of autonomous agents powered by language models (LMs) in performing decision-making tasks such as web automation. LMs excel in natural language understanding and generation but struggle with multi-step reasoning, planning, and utilizing environmental feedback when tackling realistic computer tasks. To overcome these challenges, the authors propose an inference-time search algorithm that enables LM agents to conduct exploration and multi-step planning within interactive web environments. Their approach involves implementing a best-first tree search algorithm that operates directly within the environment space. This method complements existing state-of-the-art agents and represents a novel strategy for enhancing the performance of LM agents on realistic web tasks. The authors demonstrate the effectiveness of their search algorithm by applying it to a GPT-4o agent on the VisualWebArena benchmark. The results show a significant 39.7% relative increase in success rate compared to the baseline without search, achieving a state-of-the-art success rate of 26.4%. Similarly, on WebArena, incorporating the search algorithm leads to a 28.0% relative improvement over a baseline agent and achieves a competitive success rate of 19.2%. Through extensive experiments and analysis of their results, the authors highlight the benefits of employing search algorithms for web agents and emphasize how performance scales with increased test-time compute resources. They also discuss potential limitations and promising directions for future research in this area. The code and models developed as part of this study are publicly available at https://jykoh.com/search-agents. Overall,"Tree Search for Language Model Agents" presents a valuable contribution to advancing the capabilities of LM-powered autonomous agents in complex decision-making scenarios within interactive web environments.