Summarize Paper logo SummarizePaper.com

AI summary · arXiv: 2605.18747v1 · cs.CL

Code as Agent Harness

Xuying Ning, Katherine Tieu, Dongqi Fu, Tianxin Wei, Zihao Li, Yuanchen Bei, Jiaru Zou, Mengting Ai,
and 34 more authorsZhining Liu, Ting-Wei Li, Lingjie Chen, Yanjun Zhao, Ke Yang, Bingxuan Li, Cheng Qian, Gaotang Li, Xiao Lin, Zhichen Zeng, Ruizhong Qiu, Sirui Chen, Yifan Sun, Xiyuan Yang, Ruida Wang, Rui Pan, Chenyuan Yang, Dylan Zhang, Liri Fang, Zikun Cui, Yang Cao, Pan Chen, Dorothy Sun, Ren Chen, Mahesh Srinivasan, Nipun Mathur, Yinglong Xia, Hong Li, Hong Yan, Pan Lu, Lingming Zhang, Tong Zhang, Hanghang Tong, Jingrui He
Original paper Submitted to arXiv on 18 May. 2026 GitHub: https://github.com/YennNing/Awesome-Code-as-Agent-Harness-Papers License: CC BY 4.0

How to read this summary? Illustrated guide

The essentials

  • Recent advancements in large language models (LLMs) have showcased remarkable ability to understand and generate code across various domains
  • Code is now a fundamental operational substrate for agent reasoning, action execution, environment modeling, and verification processes in emerging agentic systems
  • The concept of agent harnesses highlights code as the central component for building robust agent infrastructure
  • Harness interface facilitates connection between agents and key components like reasoning mechanisms, action execution strategies, and environment modeling techniques
  • Harness mechanisms include planning methodologies, memory management strategies, tool selection processes, feedback-driven control mechanisms for adaptability and reliability enhancement
  • Scaling the harness from single-agent setups to multi-agent environments with shared code artifacts facilitating coordination among multiple agents
  • Representative methods and practical applications of code as an agent harness span diverse domains including coding assistants, robotics applications, recommendation systems, DevOps practices, enterprise workflows
  • Open challenges in harness engineering include evaluating performance beyond task completion metrics, verifying system behavior with incomplete feedback, regression-free improvement of harness functionality, ensuring consistent shared state across multiple agents, incorporating human oversight for safety-critical actions, extending capabilities to multimodal environments

Key points written by AI: they may contain errors, check them against the paper.

AI-generated keywords: Large Language Models Agent Harnesses Code as Infrastructure Multi-Agent Environments Agentic AI Systems

Read further

  • Comprehensive Summary
  • Layman's Summary
  • Blog article
  • Original abstract
Recent advancements in large language models (LLMs) have showcased their remarkable ability to understand and generate code across various domains. From competitive programming to high-level software engineering tasks, LLMs have proven to be a powerful tool. However, in the realm of emerging agentic systems, code is no longer just a final output but rather a fundamental operational substrate that underpins agent reasoning, action execution, environment modeling, and verification processes. This paradigm shift is encapsulated through the concept of agent harnesses where code serves as the central component for building robust agent infrastructure. To delve deeper into this transformative perspective, this survey delves into three interconnected layers. Firstly, it explores the harness interface and highlights how code facilitates the connection between agents and key components such as reasoning mechanisms, action execution strategies, and environment modeling techniques. Secondly, it delves into harness mechanisms that encompass planning methodologies, memory management strategies for long-horizon execution tasks, tool selection processes for optimal performance, feedback-driven control mechanisms for adaptability and reliability enhancement. Furthermore, the survey discusses scaling the harness from single-agent setups to multi-agent environments where shared code artifacts play a crucial role in facilitating coordination among multiple agents while enabling efficient review and verification processes. The exploration of representative methods and practical applications of code as an agent harness spans diverse domains including coding assistants, GUI/OS automation tools, embodied agents in robotics applications scientific discovery processes personalization algorithms recommendation systems DevOps practices enterprise workflows. Moreover open challenges are outlined in harness engineering such as evaluating performance beyond task completion metrics verifying system behavior with incomplete feedback regression-free improvement of harness functionality ensuring consistent shared state across multiple agents incorporating human oversight for safety-critical actions extending capabilities to multimodal environments. By emphasizing code as the foundational element of agentic AI systems this survey provides a comprehensive roadmap towards developing executable verifiable and stateful AI agent systems that can effectively navigate complex real-world scenarios.
Created on 13 Jun. 2026

Was this synthesis useful?

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Ask questions about this paper to our AI assistant

How to chat with this paper? Illustrated guide

Compare with the PDF

You can also chat with multiple papers at once here.

Figures and plots

Ask about a figure by number or use its question button to request visual analysis. Up to two figures and four image panels per question. Values read from plots are approximate; check the original data.
AI assistant instructions?

Tools for this paper

Explain it differently 3 free rewrites left today
Save to a collection Group papers in collections to compare them, chat with them and export them. Create a free account Log in
Export BibTeX · RIS (Zotero, Mendeley, EndNote) · Obsidian (Markdown) · Notion (Markdown)

Advanced tools Premium

For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium

Zotero
Citation alerts Citation alerts guide
Who works on it? The people and institutions who publish on its topic
Journal club Premium+ Slides to present it Propose for the journal club
Versions What changed between versions
Talk about it in public Premium+ Press kit: release, questions, hype check Press kit guide

Papers around this one

See them on a map

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

  1. Very close Broader context
    Harness Engineering for AI Coding Agents: Emerging Practices and Principles
    Hua Zheng, Cheng Wen, Muhammad Sadiq et al. · 2026 · 2026 10th International Conference on Electronic Information Technology and Computer Engineering (EITCE) · Not summarized yet

    Harness Engineering for AI Coding Agents synthesizes industry practices for reliable coding-agent environments, while this survey frames code-based harnesses across broader agent settings.

    Abstract

    As LLM-based coding agents transition from research prototypes to production tools, practitioners are discovering that system reliability depends less on model capability than on the engineering of the surrounding environment. A nascent discipline, harness engineering, has emerged to address this gap, encompassing the design of constraints, feedback loops, context management strategies, and verification mechanisms that govern how coding agents operate within real codebases. However, current knowledge remains fragmented across blog posts, technical reports, and isolated case studies, with no unified framework to guide practitioners or researchers. In this paper, we synthesize early industry p…

  2. Very close Another approach to the problem
    Natural-Language Agent Harnesses
    Linyue Pan, Lexiao Zou, Shuo Guo et al. · 2026 · Computation and Language · preprint, not peer reviewed · cited 60 times · AI summary available

    Natural-Language Agent Harnesses encode run policies in editable documents and interpret them through a shared runtime, unlike this survey’s focus on code as harness substrate.

    Abstract

    Agent performance is strongly shaped by the surrounding harness: the external execution system around a model that organizes a task run. Yet this logic is usually buried in tightly coupled controller code, which makes harnesses hard to inspect, compare, transfer, and ablate. This paper asks whether the reusable design pattern of an agent harness can be represented as an executable natural-language object. We introduce Natural-Language Agent Harnesses (NLAHs), editable documents that describe run-level harness policy, and Intelligent Harness Runtime (IHR), a shared runtime that interprets these documents into agent calls, handoffs, state updates, validation gates, and artifact contracts. Acro…

  3. Very close Builds on this paper Cites this paper
    Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
    Ru-Han Wang, Yu-Cheng Shi, Zong-Xia Li et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 15 times · Not summarized yet

    Harness Handbook develops tools for finding and editing behavior in evolving agent harnesses, a topic within this survey’s harness engineering perspective.

    Abstract

    The capability of a modern AI agent depends not only on its foundation model but also on its harness, which constructs prompts, manages state, invokes tools, and coordinates execution. As models, APIs, environments, and requirements evolve, the harness must be continually modified. Before such a change can be made, a developer or coding agent must identify all code locations that implement the target behavior. This is difficult because production harnesses are large, tightly coupled, and behaviorally distributed, while modification requests describe what the system should do and repositories are organized by files and modules. Code search, repository indexing, and long-context processing eas…

  4. Very close Same method, another problem
    Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
    Jiahang Lin, Shichun Liu, Chengjun Pan et al. · 2026 · Computation and Language · preprint, not peer reviewed · cited 166 times · AI summary available

    Agentic Harness Engineering uses observability-driven feedback to evolve coding-agent harnesses, matching this survey’s emphasis on feedback-driven harness improvement.

    Abstract

    Harnesses are now central to coding-agent performance, mediating how models interact with tools and execution environments. Yet harness engineering remains a manual craft, because automating it faces a heterogeneous action space across editable components, voluminous trajectories that bury actionable signal, and edits whose effect is hard to attribute. We introduce Agentic Harness Engineering (AHE), a closed loop that addresses these challenges through three matched observability pillars: (1) component observability gives every editable harness component a file-level representation so the action space is explicit and revertible; (2) experience observability distills millions of raw trajector…

  5. Very close Broader context
    What makes a harness a harness: necessary and sufficient conditions for an agent harness
    Sanderson Oliveira de Macedo · 2026 · arXiv.org · preprint, not peer reviewed · cited 6 times · Not summarized yet

    What makes a harness a harness defines the boundaries of agent harnesses, while this survey examines code’s role within them.

    Abstract

    The term agent harness now circulates widely in software engineering with generative artificial intelligence. It names the layer that wraps a language model and turns it into a coding agent able to act on a repository. The usage is loose and polysemous. Sometimes the term denotes the whole product (Claude Code, Codex CLI); sometimes it denotes the evaluation scaffold that runs an agent against tasks (the SWE-bench harness); sometimes it gets conflated with an agent framework, an SDK, an IDE plugin, or an orchestrator. What is missing is a reference definition that works as an instrument, one that includes and excludes cases consistently. We build that definition through a conceptual analysis…

  6. Very close Same question Cites this paper
    Beyond the Model: Demystifying Harness Effects in Software Engineering Agents
    Hai-Chuan Hu, Quan-Jun Zhang, Sheng-Cheng Yu et al. · 2026 · cited 4 times · Not summarized yet

    Beyond the Model empirically studies how harness design affects software-engineering agents, while this survey broadly examines code-centered agent harnesses.

    Abstract

    Large Language Model (LLM)-based agents are increasingly used for software engineering tasks, yet their performance is not determined by the base model alone. The agent harness substantially shapes how SE agents interact with repositories, execute actions, and validate solutions. However, the role of harness design remains insufficiently understood, especially across different models, tasks, and harness components. In this paper, we present a systematic empirical study of harness effects in SE agents. We first evaluate two representative harnesses, mini-SWE-agent and OpenCode, with ten models from two prominent open-weight model families, Qwen and DeepSeek, on three benchmarks: SWE-bench Pro…

  7. Very close Review Cites this paper
    From Question Answering to Task Completion: A Survey on Agent System and Harness Design
    Jian-Yuan Guo, Zhi-Wei Hao, Cheng-Cheng Wang et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 15 times · Not summarized yet

    From Question Answering to Task Completion surveys agent systems and harness design through the model-harness relationship, overlapping this survey’s broad scope.

    Abstract

    LLM-based agents mark a shift from passive question answering to active task completion: they perceive environments, invoke tools, maintain state, and act over extended horizons. As agent systems have evolved from prompt engineering to workflows and context engineering, harness engineering, and agent-native training with co-evolution, a central question has become increasingly important: where does the bottleneck in agent performance reside, in the foundation model, in the execution harness, or in the coupling between them? This survey examines LLM-based agents through a model-harness lens. We first clarify the functional definition of agents and the implementation view of an LLM-based agent…

  8. Very close Same method, another problem Cites this paper Uses this paper in depth
    JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution
    Gui-Bin Zhang, Leo Lu, Fang-Zhou Xie et al. · 2026 · cited 17 times · Not summarized yet

    JIT-Agent generates task-adaptive harnesses for existing agents, addressing harness planning and adaptation highlighted in this survey.

    Abstract

    Agent capability is not determined by the model alone. The agent harness, encompassing memory management, planning strategy, action protocol, and tool/skill orchestration, can dominate the contribution of the underlying foundation model. Yet harness design remains manual, task-specific, and fundamentally unscalable. We present JIT-Agent, a harness intelligence model trained to synthesize task-adaptive agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. We formalize the agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, and train JIT-Agent to customize harnesses for a given task at hand, repair harnesses for stable and reliab…

Papers that cite it

  1. Close Same method, another problem
    Harness-Zero: Harness Distillation via Agent-as-Harness
    Hao-Ran Ye, Yuxing Lu, Hao-Nan Dong et al. · 2026 · cited 8 times · Not summarized yet

    Harness-Zero transfers behaviors from optimized harnesses into model weights, addressing how harness capabilities can be scaled beyond deployment-specific harnesses.

    Abstract

    Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment. Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. We therefore study agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so that its gains survive under a single fixed target harness. The challenge is that the two harnesses differ in action space and av…

  2. Very close Same method, another problem
    EvoHarness-RL: Learning Runtime Harness Coordination for Self-Evolving Agents
    Xu-Ying Ning, Dong-Qi Fu, Tian-Xin Wei et al. · 2026 · cited 1 time · Not summarized yet

    EvoHarness-RL learns coordination between agents and runtime harness support, addressing long-horizon execution mechanisms covered in this survey.

    Abstract

    Long-horizon LLM agents increasingly rely on external execution support to maintain state, track progress, recover from failures, and reuse experience across extended interactions. Yet existing harnesses and their use are often tailored to environments and controlled through prompts, heuristics, or system-specific rules, making agent and harness coordination difficult to jointly optimize. We introduce EvoHarness-RL, a unified framework that separates environment-specific harness implementations from a shared policy-facing interface. EvoHarness-RL organizes external support into a Belief, Progress, and Experience (BPE) workspace and exposes four compact harness actions for accessing and updat…

  3. Very close Same method, another problem
    Recurrent Harness for Agentic Continual Learning
    Yu-Xin Xiao, Zhi-Yun Lu, Jian-Yu Wang et al. · 2026 · Not summarized yet

    ReHarness revises harness state during continual learning, relating to the survey’s discussion of memory and adaptive harness mechanisms.

    Abstract

    Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after ea…

  4. Close Same method, another problem
    Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
    Min-hua Lin, Jun-Cheng Wu, Zi-Jun Wang et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 46 times · Not summarized yet

    Harness Updating Is Not Harness Benefit separates producing harness updates from benefiting from them, examining harness evolution addressed in this survey.

    Abstract

    LLM agents are increasingly deployed as systems built around editable external harnesses, including prompts, skills, memories and tools, that shape task execution without changing model parameters. Harness self-evolution adapts such agents by updating these harnesses from execution evidence. Yet it remains unclear whether a model's base capability in task-solving predicts its capabilities in harness self-evolution: which models produce useful harness updates, and which actually benefit from them? We analyze two harness self-evolution capabilities: (i) harness-updating, the capability to produce useful persistent harness updates from execution evidence; (ii) harness-benefit, the capability to…

  5. Close Shared data, tool or method
    Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution
    Zhi-Yong Cao, Heng-Jie Zhu, Yong-Xin Wang et al. · 2026 · cited 1 time · Not summarized yet

    Trial by Trace introduces executable tasks and ground truth for evaluating harness reliability, complementing this survey’s call for evaluation beyond task success.

    Abstract

    LLM agents are increasingly used to solve real-world problems, and their effectiveness depends on both the underlying models and the harnesses that support their operation. However, most existing benchmarks focus on model capabilities and overall agent performance, with limited attention to the evaluation of agent harnesses. To address this gap, we introduce TraceTrial-Bench, which organizes harness reliability into seven components covering context, execution, governance, lifecycle, observability, tools, and verification. We construct 95 expert-level, mechanism-oriented tasks with executable or state-based ground truth to support semantically equivalent comparisons while preserving the nati…

Go further

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

Updated on 31 October 2026.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.