# Tool-RoCo: An Agent-as-Tool Self-organization Large Language Model Benchmark in Multi-robot Cooperation

**Authors:** Ke Zhang, Xiaoning Zhao, Ce Zheng, Jiahong Ning, Dandan Zhu, Wenqi Zhang, Chen Sun, Toshiharu Sugawara  
**arXiv:** [2511.21510](https://arxiv.org/abs/2511.21510)  
**Date:** 2025-11-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2511.21510v1/

## Key points

- Tool-RoCo is a benchmark designed to assess large language models (LLMs) in long-term multi-agent cooperation scenarios.
- Four distinct LLM paradigms are proposed: centralized cooperation, centralized self-organization, decentralized cooperation, and self-organization.
- The benchmark includes three challenging multi-robot tasks - SORT, PACK, and CABINET - to measure format and parameter accuracy as well as agent coordination through tool utilization.
- Larger models like GPT-4.1 and GPT-5 showed stronger reflection and modification capabilities compared to smaller models such as GPT-4o-mini and GPT-5-mini.
- The latest GPT-5 model demonstrated lower reflection and modification levels despite higher tool calling accuracy and execution validity rates compared to GPT-4.1.
- Different cooperation paradigms had varying impacts on LLM performance, with Centralized Cooperation yielding stable tool usage while decentralized paradigms highlighted challenges for smaller models in distributed reasoning settings.
- Reflective adaptation and complexity within cooperative frameworks are crucial for enhancing LLM performance in multi-agent systems.
- Tool-RoCo aims to provide a comprehensive evaluation framework by exploring higher-level cooperative measures such as cooperative tool ratio (CT) and self-organization (SO).

## Summary

This study introduces Tool-RoCo, a groundbreaking benchmark designed to assess the performance of large language models (LLMs) in long-term multi-agent cooperation scenarios. Drawing inspiration from RoCo, a multi-robot cooperative benchmark, Tool-RoCo focuses on agent autonomy by treating other agents as tools and introducing cooperative tools. These tools allow each LLM to select appropriate actions based on the current state, receive feedback, and adjust their selections in subsequent rounds. To evaluate different levels of autonomy, four distinct LLM paradigms are proposed: centralized cooperation, centralized self-organization, decentralized cooperation, and self-organization. Each paradigm offers unique insights into how LLMs interact with one another and coordinate their actions through tool usage. The benchmark includes three challenging multi-robot tasks - SORT, PACK, and CABINET - aimed at measuring format and parameter accuracy as well as agent coordination within the context of tool utilization. The results obtained from various LLMs showcased intriguing patterns. Larger models such as GPT-4.1 and GPT-5 exhibited stronger reflection and modification capabilities compared to smaller models like GPT-4o-mini and GPT-5-mini. Despite achieving higher tool calling accuracy and execution validity rates, the latest GPT-5 model demonstrated lower levels of reflection and modification when compared to GPT-4.1. Furthermore, the impact of different cooperation paradigms was evident in the study's findings. The Centralized Cooperation Paradigm yielded stable tool usage due to centralized coordination by a single LLM agent. On the other hand, decentralized paradigms highlighted challenges faced by smaller models in maintaining parameter validity and execution rates under distributed reasoning settings. Overall, these results underscore the importance of reflective adaptation and complexity within cooperative frameworks for enhancing LLM performance in multi-agent systems. By moving beyond basic tool metrics to explore higher-level cooperative measures such as cooperative tool ratio (CT) and self-organization (SO), Tool-RoCo aims to provide a comprehensive evaluation framework for assessing autonomy and coordination among LLM-based agents in collaborative tasks.