AI summary · arXiv: 2606.09498v1 · cs.CL
Self-Harness: Harnesses That Improve Themselves
The essentials
- Performance of language model-based agents (LLMs) tied to base models and harnesses
- Traditional methods of human experts engineering agent harnesses inefficient and unsustainable
- Introduction of Self-Harness paradigm for LLM-based agents to autonomously enhance operating harness
- Three key stages of Self-Harness: Weakness Mining, Harness Proposal, Proposal Validation
- Implementation of Self-Harness on Terminal-Bench-2.0 with three distinct base models
- Consistent enhancement in performance metrics and increased pass rates across all three models with Self-Harness
- Qualitative analyses show transformation of weaknesses into tangible harness improvements
- Promising trajectory towards LLM-based agents actively reshaping their harnesses for greater autonomy and agility
Key points written by AI: they may contain errors, check them against the paper.
Ask questions about this paper to our AI assistant
How to chat with this paper? Illustrated guide
You can also chat with multiple papers at once here.
Keep this conversation — a free account saves your questions and answers, your collections of papers and your alerts.
Create a free accountTools for this paper
Advanced tools Premium
For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium
Papers around this one
See them on a mapRanked by how close their title and abstract are to this paper’s. Select some to read them together.
Closest papers
-
HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?Yu-Hao Wu, Jing-Yuan Zhang, Jia-Jun Shi et al. · 2026 · cited 15 times · Not summarized yet
Abstract
As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly termed the agent harness. Changing this harness while holding model weights fixed can substantially alter task performance. Current agent evaluations typically report downstream performance under a chosen harness, leaving a model's ability to develop the harness itself comparatively underexplored. We introduce HarnessDev, a benchmark that shifts the unit of evaluation from task outputs to runnable infrastructure. HarnessDev covers two stages. In Creation, the agent starts from a minimal seed and a small number of cases, then builds a complete e…
-
HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent HarnessesLuan Zhang, Ruo-Chen Zhou, Dan-Dan Song et al. · 2026 · cited 24 times · Not summarized yet
Abstract
Harness design plays a critical role in agent performance by shaping how large language models (LLMs) perceive, reason over, and act within executable environments. Recent work has proposed automatic harness evolution, which iteratively improves the harness from agent--environment interactions. However, existing methods often overfit to the evolution tasks, rely exclusively on trajectory-derived signals, and optimize harness components jointly, causing interference across components. We propose HarnessCompass, a novel automatic harness evolution framework built around constrained evolution, proactive feedback, and component-wise optimization. HarnessCompass first enforces global constraints …
-
Meta-Harness: End-to-End Optimization of Model HarnessesYoonho Lee, R. Nair, Qizheng Zhang et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 353 times · Not summarized yet
Abstract
The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively. We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management s…
-
Self-Evolving Harness on Multiple Tasks with the Agent as Its Own OptimizerQian-Kai Xu · 2026 · cited 1 time · Not summarized yet
Abstract
A harness is the code around a language-model agent that organizes prompts, calls tools, manages context, and controls execution. As models grow stronger, recent work has begun to let agents improve their own harnesses, a line of work known as self-evolving harnesses. In most existing methods, a separate proposer running on a human-designed harness modifies the solver's harness, and a separate harness is evolved for each benchmark. Real-world tasks come from many domains, so both the evolution and the evaluation of a harness should cover a diverse range of tasks. We propose a framework close to recursive self-improvement: the same frozen model, on the same version of the harness, first solve…
-
Harmony Harness: Multi-Objective Self-Evolution of Agent HarnessesWei-Cheng Yu, Lesly Miculicich, Bhavana Dalvi et al. · 2026 · Not summarized yet
Abstract
Recent work on self-evolving LLM agents treats the agent harness as an optimization target that can be iteratively improved through execution feedback. Existing approaches, however, largely optimize a single objective and evolve the harness as a monolithic artifact, limiting their ability to handle competing goals and to separate transferable design from environment-specific adaptation. We introduce Harmony Harness, a framework for multi-objective, compositional harness evolution. Harmony Harness formulates harness evolution as a multi-objective optimization problem and uses decomposition-based evolutionary search to maintain solutions across different regions of the Pareto frontier, avoidin…
-
Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent HarnessesJiahang Lin, Shichun Liu, Chengjun Pan et al. · 2026 · Computation and Language · preprint, not peer reviewed · cited 166 times · AI summary available
Abstract
Harnesses are now central to coding-agent performance, mediating how models interact with tools and execution environments. Yet harness engineering remains a manual craft, because automating it faces a heterogeneous action space across editable components, voluminous trajectories that bury actionable signal, and edits whose effect is hard to attribute. We introduce Agentic Harness Engineering (AHE), a closed loop that addresses these challenges through three matched observability pillars: (1) component observability gives every editable harness component a file-level representation so the action space is explicit and revertible; (2) experience observability distills millions of raw trajector…
-
Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM AgentsRui-Qing Yue, Yu Cui, Zhuo-Yu Sun et al. · 2026 · cited 4 times · Not summarized yet
Abstract
Self-evolving runtime harnesses can substantially improve the capabilities of large language model (LLM) agents and provide a promising paradigm for optimizing agent execution. Existing failure-driven approaches often treat observed agent failures as direct evidence for harness modification. A key challenge in failure-driven harness evolution is that observed failures can reflect either limitations of the underlying model or systematic deficiencies of the harness. Directly optimizing against individual failures can therefore induce model-specific accommodation and impair generalization across tasks and models. We study whether failure evidence accumulated across task instances can provide a …
-
Stop Comparing LLM Agents Without Disclosing the HarnessYunbei Zhang, Janet Wang, Yingqiang Ge et al. · 2026 · Artificial Intelligence · preprint, not peer reviewed · cited 44 times · AI summary available
Abstract
This position paper argues that, for long-horizon tasks evaluated across models with comparable frontier capability, the agent execution harness, namely the infrastructure layer that governs context construction, tool interaction, orchestration, and verification around a language model, is often a stronger determinant of agent performance than the model it wraps. We formalize and defend the Binding Constraint Thesis: in this regime, performance variance is governed more by harness configuration than by model choice, and current evaluation protocols therefore systematically misattribute harness-level gains to model improvements. We support this thesis along three lines. First, a control-theor…
Prior work it builds on
-
SemaClaw: A Step Towards General-Purpose Personal AI Agents through Harness EngineeringNingyan Zhu, Huacan Wang, Jie Zhou et al. · 2026 · Artificial Intelligence · preprint, not peer reviewed · cited 4 times · AI summary available
Abstract
The rise of OpenClaw in early 2026 marks the moment when millions of users began deploying personal AI agents into their daily lives, delegating tasks ranging from travel planning to multi-step research. This scale of adoption signals that two parallel arcs of development have reached an inflection point. First is a paradigm shift in AI engineering, evolving from prompt and context engineering to harness engineering-designing the complete infrastructure necessary to transform unconstrained agents into controllable, auditable, and production-reliable systems. As model capabilities converge, this harness layer is becoming the primary site of architectural differentiation. Second is the evoluti…
-
Self-Taught Optimizer (STOP): Recursively Self-Improving Code GenerationE. Zelikman, Eliana Lorch, L. Mackey et al. · 2023 · arXiv.org · preprint, not peer reviewed · cited 217 times · AI summary available
Abstract
Several recent advances in AI systems solve problems by providing a"scaffolding"program that structures multiple calls to language models (LMs) to generate better outputs. A scaffolding program is written in a programming language such as Python. In this work, we use a language-model-infused scaffolding program to improve itself. We start with a seed"improver"that improves an input program according to a given utility function by querying an LM several times and returning the best solution. We then run this seed improver to improve itself. Across a small set of downstream tasks, the resulting improved improver generates programs with significantly better performance than its seed improver. A…
-
Language Agents as Optimizable GraphsMing-Chen Zhuge, Wen-Yi Wang, Louis Kirsch et al. · 2024 · International Conference on Machine Learning · cited 89 times · Not summarized yet
Abstract
Various human-designed prompt engineering techniques have been proposed to improve problem solvers based on Large Language Models (LLMs), yielding many disparate code bases. We unify these approaches by describing LLM-based agents as computational graphs. The nodes implement functions to process multimodal data or query LLMs, and the edges describe the information flow between operations. Graphs can be recursively combined into larger composite graphs representing hierarchies of inter-agent collaboration (where edges connect operations of different agents). Our novel automatic graph optimizers (1) refine node-level LLM prompts (node optimization) and (2) improve agent orchestration by changi…
-
The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic FeedbackHang-Fan Zhang, Si-Yuan Xu, Zhimeng Guo et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 5 times · Not summarized yet
Abstract
Reinforcement learning (RL) has demonstrated potential in enhancing the reasoning capabilities of large language models (LLMs), but such training typically demands substantial efforts in creating and annotating data. In this work, we explore improving LLMs through RL with minimal data. Our approach alternates between the LLM proposing a task and then attempting to solve it. To minimize data dependency, we introduce two novel mechanisms grounded in self-awareness: (1) self-aware difficulty prediction, where the model learns to assess task difficulty relative to its own abilities and prioritize challenging yet solvable tasks, and (2) self-aware limit breaking, where the model recognizes when a…
-
Agentic Context Engineering: Evolving Contexts for Self-Improving Language ModelsQizheng Zhang, Chang-Ran Hu, Shubhangi Upasani et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 500 times · Not summarized yet
Abstract
Large language model (LLM) applications such as agents and domain-specific reasoning increasingly rely on context adaptation: modifying inputs with instructions, strategies, or evidence, rather than weight updates. Prior approaches improve usability but often suffer from brevity bias, which drops domain insights for concise summaries, and from context collapse, where iterative rewriting erodes details over time. We introduce ACE (Agentic Context Engineering), a framework that treats contexts as evolving playbooks that accumulate, refine, and organize strategies through a modular process of generation, reflection, and curation. ACE prevents collapse with structured, incremental updates that p…
Papers that cite it
-
Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM AgentsTian-Shi Xu, Huifeng Wen, Meng Li · 2026 · arXiv.org · preprint, not peer reviewed · cited 32 times · Not summarized yet
Abstract
LLM agents are shaped not only by their language models, but also by the runtime harness that mediates observation, tool use, action execution, feedback interpretation, and trajectory control. While existing agent adaptation methods mainly update model parameters, many failures in deterministic, rule-governed domains stem from mismatches at the model--environment interface. We propose Life-Harness, a lifecycle-aware runtime harness that improves frozen LLM agents without changing model weights or evaluation environments. Life-Harness evolves from training trajectories by converting recurring interaction failures into reusable interventions across environment contracts, procedural skills, act…
-
Harness-Zero: Harness Distillation via Agent-as-HarnessHao-Ran Ye, Yuxing Lu, Hao-Nan Dong et al. · 2026 · cited 8 times · Not summarized yet
Abstract
Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment. Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. We therefore study agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so that its gains survive under a single fixed target harness. The challenge is that the two harnesses differ in action space and av…
-
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness EvolutionGui-Bin Zhang, Leo Lu, Fang-Zhou Xie et al. · 2026 · cited 17 times · Not summarized yet
Abstract
Agent capability is not determined by the model alone. The agent harness, encompassing memory management, planning strategy, action protocol, and tool/skill orchestration, can dominate the contribution of the underlying foundation model. Yet harness design remains manual, task-specific, and fundamentally unscalable. We present JIT-Agent, a harness intelligence model trained to synthesize task-adaptive agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. We formalize the agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, and train JIT-Agent to customize harnesses for a given task at hand, repair harnesses for stable and reliab…
-
Evo-Bench: Can Language Models Improve Agent Harness?Lisheng Huang, Chen Yang, Hao Zhou et al. · 2026 · cited 12 times · Not summarized yet
Abstract
Large Language Models (LLMs) have driven rapid progress in autonomous agents, yet standard evaluations remain confined to static task solving. An emerging frontier is harness evolution---the agent's capacity to autonomously optimize its own operating harness. However, systematically benchmarking this capability remains challenging, as existing evaluations fail to isolate harness improvements from base model strength, prevent task-specific overfitting, or capture long-horizon iterative research. To address these challenges, we introduce Evo-Bench, the first benchmark designed to evaluate models'intrinsic harness-evolving capabilities across Search, Office, and General agent domains. To rigoro…
-
Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent HarnessesTailin Zhou · 2026 · cited 9 times · Not summarized yet
Abstract
Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the \emph{harness}---is typically treated as a fixed artifact after deployment. This work studies an alternative where the harness is \emph{task-specific and continuously evolvable}: each task family maintains its own harness, which is hot-swapped across iterations through a fixed task-injection seam and rewritten using environment feedback. We introduce \textbf{Hierarchical Self-Improvement (HSI)}, a framework in which a single frozen LLM $M$ operates across three hierarchical scopes: a task harness $H$ that executes tasks, an evolver that rewrites $…
Go further
How are these papers found?
We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.
Updated on 31 October 2026.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.