Natural-Language Agent Harnesses

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Same question Not relevant

   Self-Harness: Harnesses That Improve Themselves

   Hangfan Zhang, Shao Zhang, Kangcong Li et al. · 2026 · Computation and Language · preprint, not peer reviewed · cited 123 times · AI summary available

   Self-Harness studies how agent harnesses shape performance and proposes an LLM agent that improves its own harness.

   Abstract

   The performance of LLM-based agents is jointly shaped by their base models and the harnesses that mediate their interaction with the environment. Because different models exhibit distinct behaviors, effective harness design is inherently model-specific. Yet agent harnesses are still largely engineered by human experts, a paradigm that scales poorly as modern LLMs become increasingly diverse and rapidly evolving. In this paper, we introduce Self-Harness, a new paradigm in which an LLM-based agent improves its own operating harness, without relying on human engineers or stronger external agents. We operationalize Self-Harness as an iterative loop with three stages: Weakness Mining, which ident…

2. Very close Same question Cites this paper Not relevant

   MemoHarness: Agent Harnesses That Learn from Experience

   Yue Huang, Wen-Jie Wang, Han Bao et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 18 times · Not summarized yet

   MemoHarness adapts editable harness controls from execution experience, while this paper represents harness policy in executable natural-language documents.

   Abstract

   An agent harness is the external control layer that turns a base LLM into an executable agent by managing context, tools, orchestration, memory, decoding, and output handling. While harness design strongly affects agent behavior, most automatic improvement methods optimize narrower artifacts such as prompts, pipelines, or workflows, and deployed agents usually reuse a single global harness for all cases. We introduce MemoHarness, an adaptive harness optimization framework that learns from its own executions. MemoHarness decomposes the harness into six editable control dimensions, stores per-case diagnoses and distilled global patterns in a dual-layer experience bank, and adapts the learned h…

3. Very close Broader context Not relevant

   Code as Agent Harness

   Xuying Ning, Katherine Tieu, Dongqi Fu et al. · 2026 · Computation and Language · preprint, not peer reviewed · cited 54 times · AI summary available

   Code as Agent Harness surveys code as the operational substrate for agent infrastructure, whereas this paper represents harness policies as natural-language documents.

   Abstract

   Recent large language models (LLMs) have demonstrated strong capabilities in understanding and generating code, from competitive programming to repository-level software engineering. In emerging agentic systems, code is no longer only a target output. It increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification. We frame this shift through the lens of agent harnesses and introduce code as agent harness: a unified view that centers code as the basis for agent infrastructure. To systematically study this perspective, we organize the survey around three connected layers. First, we study the harness interface, where code c…

4. Very close Same question Not relevant

   Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses

   Jiahang Lin, Shichun Liu, Chengjun Pan et al. · 2026 · Computation and Language · preprint, not peer reviewed · cited 166 times · AI summary available

   Agentic Harness Engineering automatically evolves coding-agent harness components using observability, while this paper makes harness policies explicit and analyzable.

   Abstract

   Harnesses are now central to coding-agent performance, mediating how models interact with tools and execution environments. Yet harness engineering remains a manual craft, because automating it faces a heterogeneous action space across editable components, voluminous trajectories that bury actionable signal, and edits whose effect is hard to attribute. We introduce Agentic Harness Engineering (AHE), a closed loop that addresses these challenges through three matched observability pillars: (1) component observability gives every editable harness component a file-level representation so the action space is explicit and revertible; (2) experience observability distills millions of raw trajector…

5. Very close Same question Cites this paper Not relevant

   JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution

   Gui-Bin Zhang, Leo Lu, Fang-Zhou Xie et al. · 2026 · cited 17 times · Not summarized yet

   JIT-Agent synthesizes task-adaptive harnesses for agents, while this paper uses natural-language documents to represent executable harness policies.

   Abstract

   Agent capability is not determined by the model alone. The agent harness, encompassing memory management, planning strategy, action protocol, and tool/skill orchestration, can dominate the contribution of the underlying foundation model. Yet harness design remains manual, task-specific, and fundamentally unscalable. We present JIT-Agent, a harness intelligence model trained to synthesize task-adaptive agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. We formalize the agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, and train JIT-Agent to customize harnesses for a given task at hand, repair harnesses for stable and reliab…

6. Very close Broader context Cites this paper Not relevant

   Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable

   Ru-Han Wang, Yu-Cheng Shi, Zong-Xia Li et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 15 times · Not summarized yet

   Harness Handbook addresses finding and editing behavior in coupled harness code, while this paper makes harness policy readable as natural-language documents.

   Abstract

   The capability of a modern AI agent depends not only on its foundation model but also on its harness, which constructs prompts, manages state, invokes tools, and coordinates execution. As models, APIs, environments, and requirements evolve, the harness must be continually modified. Before such a change can be made, a developer or coding agent must identify all code locations that implement the target behavior. This is difficult because production harnesses are large, tightly coupled, and behaviorally distributed, while modification requests describe what the system should do and repositories are organized by files and modules. Code search, repository indexing, and long-context processing eas…

7. Very close Same question Not relevant

   Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows

   Yilun Yao, Xinyu Tan, Chao-Hsuan Liu et al. · 2026 · Artificial Intelligence · cited 3 times · AI summary available

   Harness-Bench measures how harness configurations affect agent workflows, while this paper evaluates natural-language harnesses across agent benchmarks.

   Abstract

   LLM agents are increasingly deployed as executable systems that use tools, modify workspaces, and produce concrete artifacts. In such workflows, performance depends not only on the base model, but also on the harness: the system layer that manages context, tools, state, constraints, permissions, tracing, and recovery. However, existing benchmarks typically abstract away execution, compare complete agent systems, or hold the harness fixed, making execution-layer variation difficult to study. We introduce Harness-Bench, a diagnostic benchmark for evaluating configuration-level harness effects in realistic agent workflows. Harness-Bench evaluates representative harness configurations across mul…

8. Very close Same question Cites this paper Not relevant

   Beyond the Model: Demystifying Harness Effects in Software Engineering Agents

   Hai-Chuan Hu, Quan-Jun Zhang, Sheng-Cheng Yu et al. · 2026 · cited 4 times · Not summarized yet

   Beyond the Model empirically studies harness effects in software-engineering agents, while this paper tests natural-language harnesses across coding and other tasks.

   Abstract

   Large Language Model (LLM)-based agents are increasingly used for software engineering tasks, yet their performance is not determined by the base model alone. The agent harness substantially shapes how SE agents interact with repositories, execute actions, and validate solutions. However, the role of harness design remains insufficiently understood, especially across different models, tasks, and harness components. In this paper, we present a systematic empirical study of harness effects in SE agents. We first evaluate two representative harnesses, mini-SWE-agent and OpenCode, with ten models from two prominent open-weight model families, Qwen and DeepSeek, on three benchmarks: SWE-bench Pro…

Prior work it builds on

1. Very close Same method, another problem Not relevant

   Meta-Harness: End-to-End Optimization of Model Harnesses

   Yoonho Lee, R. Nair, Qizheng Zhang et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 353 times · Not summarized yet

   Meta-Harness searches over harness code using execution feedback, while this paper uses an intelligent runtime to execute natural-language harness descriptions.

   Abstract

   The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively. We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management s…

2. Very close Same method, another problem Not relevant

   AutoHarness: improving LLM agents by automatically synthesizing a code harness

   Xinghua Lou, Miguel Lázaro-Gredilla, Antoine Dedieu et al. · 2026 · Computation and Language · preprint, not peer reviewed · cited 82 times · AI summary available

   AutoHarness iteratively synthesizes code harnesses from environment feedback, while this paper executes harness policies written in natural language.

   Abstract

   Despite significant strides in language models in the last few years, when used as agents, such models often try to perform actions that are not just suboptimal for a given state, but are strictly prohibited by the external environment. For example, in the recent Kaggle GameArena chess competition, 78% of Gemini-2.5-Flash losses were attributed to illegal moves. Often people manually write "harnesses" around LLMs to prevent such failures. In this paper, we demonstrate that Gemini-2.5-Flash can automatically synthesize such a code harness, using a small number of rounds of iterative code refinement given feedback from the (game) environment. The resulting harness prevents all illegal moves in…

3. Close Same method, another problem Not relevant

   General Modular Harness for LLM Agents in Multi-Turn Gaming Environments

   Yuxuan Zhang, Haoyang Yu, Lan-Xiang Hu et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 9 times · Not summarized yet

   General Modular Harness evaluates perception, memory, and reasoning modules in gaming agents, while this paper tests explicit harness modules across task types.

   Abstract

   We introduce a modular harness design for LLM agents that composes of perception, memory, and reasoning components, enabling a single LLM or VLM backbone to tackle a wide spectrum of multi turn gaming environments without domain-specific engineering. Using classic and modern game suites as low-barrier, high-diversity testbeds, our framework provides a unified workflow for analyzing how each module affects performance across dynamic interactive settings. Extensive experiments demonstrate that the harness lifts gameplay performance consistently over un-harnessed baselines and reveals distinct contribution patterns, for example, memory dominates in long-horizon puzzles while perception is criti…

4. Related Broader context Not relevant

   Executable Code Actions Elicit Better LLM Agents

   Xingyao Wang, Yangyi Chen, Lifan Yuan et al. · 2024 · Computation and Language · cited 751 times · AI summary available

   Executable Code Actions uses generated Python as an agent action space, whereas this paper uses natural language to specify the surrounding harness policy.

   Abstract

   Large Language Model (LLM) agents, capable of performing a broad range of actions, such as invoking tools and controlling robots, show great potential in tackling real-world challenges. LLM agents are typically prompted to produce actions by generating JSON or text in a pre-defined format, which is usually limited by constrained action space (e.g., the scope of pre-defined tools) and restricted flexibility (e.g., inability to compose multiple tools). This work proposes to use executable Python code to consolidate LLM agents' actions into a unified action space (CodeAct). Integrated with a Python interpreter, CodeAct can execute code actions and dynamically revise prior actions or emit new ac…

5. Related Same method, another problem Not relevant

   Agint: Agentic Graph Compilation for Software Engineering Agents

   A. Chivukula, Jay Somasundaram, VijayaSai Somasundaram · 2025 · arXiv.org · preprint, not peer reviewed · cited 6 times · Not summarized yet

   Agint compiles natural-language instructions into executable graphs with a runtime, while this paper interprets natural-language harness documents into agent operations.

   Abstract

   LLM-based coding agents are increasingly common but still face challenges in context management, latency, reliability, reproducibility, and scalability. We present Agint, an agentic graph compiler, interpreter, and runtime that incrementally and hierarchically converts natural-language instructions into typed, effect-aware code DAGs. Agint introduces explicit type floors (text to data to spec to code) grounded in semantic graph transformations and a hybrid LLM and function-based JIT runtime. This enables dynamic graph refinement, reproducible and optimizable execution, speculative evaluation, and interoperability with existing developer tools. Agint's typed graph bindings improve reliability…

Papers that cite it

1. Very close Review Uses this paper in depth Not relevant

   From Question Answering to Task Completion: A Survey on Agent System and Harness Design

   Jian-Yuan Guo, Zhi-Wei Hao, Cheng-Cheng Wang et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 15 times · Not summarized yet

   From Question Answering to Task Completion surveys agent systems through the model-harness relationship, including harness engineering and evolution.

   Abstract

   LLM-based agents mark a shift from passive question answering to active task completion: they perceive environments, invoke tools, maintain state, and act over extended horizons. As agent systems have evolved from prompt engineering to workflows and context engineering, harness engineering, and agent-native training with co-evolution, a central question has become increasingly important: where does the bottleneck in agent performance reside, in the foundation model, in the execution harness, or in the coupling between them? This survey examines LLM-based agents through a model-harness lens. We first clarify the functional definition of agents and the implementation view of an LLM-based agent…

2. Very close Same question Not relevant

   Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents

   Min-hua Lin, Jun-Cheng Wu, Zi-Jun Wang et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 46 times · Not summarized yet

   Harness Updating Is Not Harness Benefit examines harness self-evolution capabilities, while this paper studies harnesses as explicit, editable representations.

   Abstract

   LLM agents are increasingly deployed as systems built around editable external harnesses, including prompts, skills, memories and tools, that shape task execution without changing model parameters. Harness self-evolution adapts such agents by updating these harnesses from execution evidence. Yet it remains unclear whether a model's base capability in task-solving predicts its capabilities in harness self-evolution: which models produce useful harness updates, and which actually benefit from them? We analyze two harness self-evolution capabilities: (i) harness-updating, the capability to produce useful persistent harness updates from execution evidence; (ii) harness-benefit, the capability to…

3. Very close Same question Not relevant

   From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws

   Meng-Zhuo Chen, Junjie Wang, Zhe Liu et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 36 times · Not summarized yet

   HarnessFix diagnoses and repairs harness flaws from failed trajectories, while this paper exposes harness modules for analysis and ablation.

   Abstract

   LLM agents increasingly rely on agent harness: the runtime infrastructure around the base model that defines execution environments, tool interfaces, context, lifecycle orchestration, observability, verification, and governance. Existing self-improving agents and automatic harness evolution methods mainly improve agents through runtime supervision, prompt optimization, workflow search, or harness modification based on final outcomes. However, they often fail to diagnose where the responsible evidence lies in failed trajectories and which harness implementation mechanism causes the unreliable behavior, resulting in broad, indirect, or poorly scoped changes. This paper proposes HarnessFix, a t…

4. Very close Same method, another problem Not relevant

   EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents

   Nanxi Li, Ying-Zi Ma, Yu-Long Cao et al. · 2026 · cited 4 times · Not summarized yet

   EvoSafeHarness evolves natural-language policies alongside enforcement logic, while this paper executes natural-language descriptions of general harness policy.

   Abstract

   Large language model (LLM) agents can translate natural-language requests into real-world actions, creating safety risks from both indirect prompt injections and directly harmful requests. Existing system-level safety harnesses are typically designed by experts and applied across heterogeneous models and domains, despite substantial differences in model behavior, utility tolerance, and domain-specific safety requirements. We present EvoSafeHarness, a framework for automatically synthesizing deployment-specific safety harnesses for frozen LLM agents. Given a target model and domain specification, EvoSafeHarness jointly evolves a natural-language policy and executable enforcement logic using b…

5. Very close Same question Not relevant

   Harnesses for Inference-Time Alignment over Execution Trajectories

   Bo-Yuan Wang, Bochao Li, Ming-Han Wang et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 3 times · Not summarized yet

   Harnesses for Inference-Time Alignment studies how harness design changes task execution and success, while this paper evaluates explicit harness policies and their effects.

   Abstract

   Harness engineering has emerged as an important inference-time technique for large language model (LLM) agents, aiming to improve long-term performance through task decomposition and guided execution. However, more elaborate harnesses are not uniformly better: increasing decomposition or guidance can sometimes improve execution, but can also reduce final task success. We study harness design through the lens of inference-time trajectory alignment. This perspective separates harness into two mechanisms: task decomposition, which structures a task into sub-goals, and guided execution, which reshapes local action distributions during execution. This decomposition allows us to quantify how workf…