Results of the summarizing process for the arXiv paper: 2606.10209v1

In their study titled "Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents," authors Abhilasha Lodha, Mahsa Pahlavikhah Varnosfaderani, Abir Chakraborty, and Abhinav Mithal address the challenges faced by large language models deployed as autonomous agents in enterprise workflows. They focus on the issue of verbose tool responses from enterprise systems leading to context overflow, stale-state errors, and high inference cost specifically in automated expense itemization within Microsoft Dynamics 365 Finance and Operations using Model Context Protocol tools. The research evaluates four different configurations of GPT-5 on a 50-task hotel expense benchmark. These configurations include no user model, full conversation history retention, context pruned to the last 5 tool call/response pairs, and pruning with automated summarization. The results are based on averaging across 5 independent runs while keeping the user model constant for comparison in context engineering. The baseline scenario without a user model achieves only 8.0% complete itemization. Retaining full conversation history improves completion to 71.0%, but at a significant cost of consuming 1,480,996 tokens and taking 14.56 hours per benchmark. Pruning the context to only the last 5 tool calls enhances completion to 79.0%, while reducing token usage to 535,274 and runtime to 5.39 hours. The most successful configuration involves adding automated summarization to pruning recent interactions which achieves a completion rate of 91.6% for itemization and an average amount itemized of 99.64%. This is accomplished with a token use of 553,374 and a runtime of 5.79 hours. Additionally, the study reports confidence intervals, effect-size analysis, sensitivity over pruning and summary windows, failure analysis, results across five expense types grouped into three categories, and cross-model evidence with Claude Sonnet 4.5. Overall, the findings demonstrate that selective retention of recent tool interactions combined with compact summarization can significantly enhance both reliability and efficiency compared to retaining full-history context in this class of enterprise tool-use workflows.

• - Authors address challenges faced by large language models deployed as autonomous agents in enterprise workflows
• - Focus on verbose tool responses leading to context overflow, stale-state errors, and high inference cost in automated expense itemization within Microsoft Dynamics 365 Finance and Operations using Model Context Protocol tools
• - Evaluation of four different configurations of GPT-5 on a 50-task hotel expense benchmark
• - No user model achieves only 8.0% complete itemization
• - Retaining full conversation history improves completion to 71.0%, but at a significant cost
• - Pruning context to last 5 tool call/response pairs enhances completion to 79.0%
• - Adding automated summarization to pruning recent interactions achieves a completion rate of 91.6% for itemization and an average amount itemized of 99.64%
• - Selective retention of recent tool interactions combined with compact summarization significantly enhances reliability and efficiency compared to retaining full-history context

SummaryAuthors are talking about problems faced by big language models used as independent workers in businesses. They focus on long tool responses causing issues like too much information, old mistakes, and high costs when organizing expenses in Microsoft Dynamics 365 Finance and Operations. They tested different versions of a model called GPT-5 for a hotel expense task. None of the models could complete all the tasks perfectly, but one that kept track of recent conversations did the best at 91.6% completion rate. Definitions- Authors: People who write books or articles. - Language models: Programs that can understand and generate human-like text. - Autonomous agents: Machines or software that can work independently without constant human input. - Enterprise workflows: The way tasks are organized and completed within a business. - Inference cost: The resources needed to make decisions based on available information.

Introduction

In recent years, large language models (LLMs) have shown remarkable capabilities in natural language processing tasks. These models are being increasingly deployed as autonomous agents in enterprise workflows, where they interact with users and perform various tasks. However, these LLM agents face challenges when dealing with verbose tool responses from enterprise systems. This leads to context overflow, stale-state errors, and high inference costs. To address these challenges, a team of researchers from Microsoft Research India conducted a study titled "Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents." The paper was authored by Abhilasha Lodha, Mahsa Pahlavikhah Varnosfaderani, Abir Chakraborty, and Abhinav Mithal. The study focuses on the specific issue of automated expense itemization within Microsoft Dynamics 365 Finance and Operations using Model Context Protocol tools. It evaluates different configurations of GPT-5 on a 50-task hotel expense benchmark to determine the most efficient approach for context engineering in this type of workflow.

The Problem

Enterprise workflows often involve complex interactions between users and systems. In the case of automated expense itemization within Microsoft Dynamics 365 Finance and Operations using Model Context Protocol tools, the user inputs information about their expenses while the system processes this information to generate an itemized report. However, due to verbose tool responses from the system during these interactions, there is an overflow of context which can lead to errors and high inference costs. This makes it challenging for LLM agents to accurately complete their task without consuming excessive resources.

The Solution

To address this problem, the researchers propose selective retention of recent tool interactions combined with compact summarization as an efficient approach for context engineering in long-horizon tool-using LLM agents. They evaluate four different configurations of GPT-5 on a 50-task hotel expense benchmark. These configurations include no user model, full conversation history retention, context pruned to the last 5 tool call/response pairs, and pruning with automated summarization.

The Results

The results of the study show that the most successful configuration involves adding automated summarization to pruning recent interactions. This approach achieves a completion rate of 91.6% for itemization and an average amount itemized of 99.64%. It also significantly reduces token usage to 553,374 and runtime to 5.79 hours. Compared to the baseline scenario without a user model which only achieves an 8.0% completion rate, this is a significant improvement in both reliability and efficiency.

Additional Findings

Apart from evaluating different configurations of GPT-5, the study also reports confidence intervals, effect-size analysis, sensitivity over pruning and summary windows, failure analysis, results across five expense types grouped into three categories, and cross-model evidence with Claude Sonnet 4.5. These findings provide further insights into the effectiveness of selective retention and compact summarization in improving LLM agent performance in enterprise workflows.

Conclusion

In conclusion, the research paper "Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents" presents a comprehensive evaluation of different approaches for context engineering in long-horizon tool-using LLM agents deployed as autonomous agents in enterprise workflows. The findings demonstrate that selective retention of recent tool interactions combined with compact summarization can significantly enhance both reliability and efficiency compared to retaining full-history context. This has important implications for improving LLM agent performance in various real-world applications where verbose tool responses are common.