Self-Harness: Harnesses That Improve Themselves

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Très proche Cite cet article Pas pertinent

   HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?

   Yu-Hao Wu, Jing-Yuan Zhang, Jia-Jun Shi et al. · 2026 · cité 15 fois · Pas encore résumé

   Résumé scientifique

   As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly termed the agent harness. Changing this harness while holding model weights fixed can substantially alter task performance. Current agent evaluations typically report downstream performance under a chosen harness, leaving a model's ability to develop the harness itself comparatively underexplored. We introduce HarnessDev, a benchmark that shifts the unit of evaluation from task outputs to runnable infrastructure. HarnessDev covers two stages. In Creation, the agent starts from a minimal seed and a small number of cases, then builds a complete e…

2. Très proche Cite cet article Pas pertinent

   HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses

   Luan Zhang, Ruo-Chen Zhou, Dan-Dan Song et al. · 2026 · cité 24 fois · Pas encore résumé

   Résumé scientifique

   Harness design plays a critical role in agent performance by shaping how large language models (LLMs) perceive, reason over, and act within executable environments. Recent work has proposed automatic harness evolution, which iteratively improves the harness from agent--environment interactions. However, existing methods often overfit to the evolution tasks, rely exclusively on trajectory-derived signals, and optimize harness components jointly, causing interference across components. We propose HarnessCompass, a novel automatic harness evolution framework built around constrained evolution, proactive feedback, and component-wise optimization. HarnessCompass first enforces global constraints …

3. Très proche Cité par cet article Utilisé en profondeur par cet article Pas pertinent

   Meta-Harness: End-to-End Optimization of Model Harnesses

   Yoonho Lee, R. Nair, Qizheng Zhang et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 353 fois · Pas encore résumé

   Résumé scientifique

   The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively. We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management s…

4. Très proche Cite cet article Pas pertinent

   Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer

   Qian-Kai Xu · 2026 · cité 1 fois · Pas encore résumé

   Résumé scientifique

   A harness is the code around a language-model agent that organizes prompts, calls tools, manages context, and controls execution. As models grow stronger, recent work has begun to let agents improve their own harnesses, a line of work known as self-evolving harnesses. In most existing methods, a separate proposer running on a human-designed harness modifies the solver's harness, and a separate harness is evolved for each benchmark. Real-world tasks come from many domains, so both the evolution and the evaluation of a harness should cover a diverse range of tasks. We propose a framework close to recursive self-improvement: the same frozen model, on the same version of the harness, first solve…

5. Très proche Cite cet article Pas pertinent

   Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses

   Wei-Cheng Yu, Lesly Miculicich, Bhavana Dalvi et al. · 2026 · Pas encore résumé

   Résumé scientifique

   Recent work on self-evolving LLM agents treats the agent harness as an optimization target that can be iteratively improved through execution feedback. Existing approaches, however, largely optimize a single objective and evolve the harness as a monolithic artifact, limiting their ability to handle competing goals and to separate transferable design from environment-specific adaptation. We introduce Harmony Harness, a framework for multi-objective, compositional harness evolution. Harmony Harness formulates harness evolution as a multi-objective optimization problem and uses decomposition-based evolutionary search to maintain solutions across different regions of the Pareto frontier, avoidin…

6. Très proche Cité par cet article Pas pertinent

   Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses

   Jiahang Lin, Shichun Liu, Chengjun Pan et al. · 2026 · Computation and Language · preprint, non relu par les pairs · cité 166 fois · Résumé IA disponible

   Résumé scientifique

   Harnesses are now central to coding-agent performance, mediating how models interact with tools and execution environments. Yet harness engineering remains a manual craft, because automating it faces a heterogeneous action space across editable components, voluminous trajectories that bury actionable signal, and edits whose effect is hard to attribute. We introduce Agentic Harness Engineering (AHE), a closed loop that addresses these challenges through three matched observability pillars: (1) component observability gives every editable harness component a file-level representation so the action space is explicit and revertible; (2) experience observability distills millions of raw trajector…

7. Très proche Cite cet article Pas pertinent

   Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents

   Rui-Qing Yue, Yu Cui, Zhuo-Yu Sun et al. · 2026 · cité 4 fois · Pas encore résumé

   Résumé scientifique

   Self-evolving runtime harnesses can substantially improve the capabilities of large language model (LLM) agents and provide a promising paradigm for optimizing agent execution. Existing failure-driven approaches often treat observed agent failures as direct evidence for harness modification. A key challenge in failure-driven harness evolution is that observed failures can reflect either limitations of the underlying model or systematic deficiencies of the harness. Directly optimizing against individual failures can therefore induce model-specific accommodation and impair generalization across tasks and models. We study whether failure evidence accumulated across task instances can provide a …

8. Très proche Pas pertinent

   Stop Comparing LLM Agents Without Disclosing the Harness

   Yunbei Zhang, Janet Wang, Yingqiang Ge et al. · 2026 · Artificial Intelligence · preprint, non relu par les pairs · cité 44 fois · Résumé IA disponible

   Résumé scientifique

   This position paper argues that, for long-horizon tasks evaluated across models with comparable frontier capability, the agent execution harness, namely the infrastructure layer that governs context construction, tool interaction, orchestration, and verification around a language model, is often a stronger determinant of agent performance than the model it wraps. We formalize and defend the Binding Constraint Thesis: in this regime, performance variance is governed more by harness configuration than by model choice, and current evaluation protocols therefore systematically misattribute harness-level gains to model improvements. We support this thesis along three lines. First, a control-theor…

Les travaux antérieurs sur lesquels il s’appuie

1. Proche Pas pertinent

   SemaClaw: A Step Towards General-Purpose Personal AI Agents through Harness Engineering

   Ningyan Zhu, Huacan Wang, Jie Zhou et al. · 2026 · Artificial Intelligence · preprint, non relu par les pairs · cité 4 fois · Résumé IA disponible

   Résumé scientifique

   The rise of OpenClaw in early 2026 marks the moment when millions of users began deploying personal AI agents into their daily lives, delegating tasks ranging from travel planning to multi-step research. This scale of adoption signals that two parallel arcs of development have reached an inflection point. First is a paradigm shift in AI engineering, evolving from prompt and context engineering to harness engineering-designing the complete infrastructure necessary to transform unconstrained agents into controllable, auditable, and production-reliable systems. As model capabilities converge, this harness layer is becoming the primary site of architectural differentiation. Second is the evoluti…

2. Apparenté Pas pertinent

   Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation

   E. Zelikman, Eliana Lorch, L. Mackey et al. · 2023 · arXiv.org · preprint, non relu par les pairs · cité 217 fois · Résumé IA disponible

   Résumé scientifique

   Several recent advances in AI systems solve problems by providing a"scaffolding"program that structures multiple calls to language models (LMs) to generate better outputs. A scaffolding program is written in a programming language such as Python. In this work, we use a language-model-infused scaffolding program to improve itself. We start with a seed"improver"that improves an input program according to a given utility function by querying an LM several times and returning the best solution. We then run this seed improver to improve itself. Across a small set of downstream tasks, the resulting improved improver generates programs with significantly better performance than its seed improver. A…

3. Apparenté Pas pertinent

   Language Agents as Optimizable Graphs

   Ming-Chen Zhuge, Wen-Yi Wang, Louis Kirsch et al. · 2024 · International Conference on Machine Learning · cité 89 fois · Pas encore résumé

   Résumé scientifique

   Various human-designed prompt engineering techniques have been proposed to improve problem solvers based on Large Language Models (LLMs), yielding many disparate code bases. We unify these approaches by describing LLM-based agents as computational graphs. The nodes implement functions to process multimodal data or query LLMs, and the edges describe the information flow between operations. Graphs can be recursively combined into larger composite graphs representing hierarchies of inter-agent collaboration (where edges connect operations of different agents). Our novel automatic graph optimizers (1) refine node-level LLM prompts (node optimization) and (2) improve agent orchestration by changi…

4. Apparenté Pas pertinent

   The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback

   Hang-Fan Zhang, Si-Yuan Xu, Zhimeng Guo et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 5 fois · Pas encore résumé

   Résumé scientifique

   Reinforcement learning (RL) has demonstrated potential in enhancing the reasoning capabilities of large language models (LLMs), but such training typically demands substantial efforts in creating and annotating data. In this work, we explore improving LLMs through RL with minimal data. Our approach alternates between the LLM proposing a task and then attempting to solve it. To minimize data dependency, we introduce two novel mechanisms grounded in self-awareness: (1) self-aware difficulty prediction, where the model learns to assess task difficulty relative to its own abilities and prioritize challenging yet solvable tasks, and (2) self-aware limit breaking, where the model recognizes when a…

5. Apparenté Pas pertinent

   Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models

   Qizheng Zhang, Chang-Ran Hu, Shubhangi Upasani et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 500 fois · Pas encore résumé

   Résumé scientifique

   Large language model (LLM) applications such as agents and domain-specific reasoning increasingly rely on context adaptation: modifying inputs with instructions, strategies, or evidence, rather than weight updates. Prior approaches improve usability but often suffer from brevity bias, which drops domain insights for concise summaries, and from context collapse, where iterative rewriting erodes details over time. We introduce ACE (Agentic Context Engineering), a framework that treats contexts as evolving playbooks that accumulate, refine, and organize strategies through a modular process of generation, reflection, and curation. ACE prevents collapse with structured, incremental updates that p…

Les articles qui le citent

1. Très proche Pas pertinent

   Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents

   Tian-Shi Xu, Huifeng Wen, Meng Li · 2026 · arXiv.org · preprint, non relu par les pairs · cité 32 fois · Pas encore résumé

   Résumé scientifique

   LLM agents are shaped not only by their language models, but also by the runtime harness that mediates observation, tool use, action execution, feedback interpretation, and trajectory control. While existing agent adaptation methods mainly update model parameters, many failures in deterministic, rule-governed domains stem from mismatches at the model--environment interface. We propose Life-Harness, a lifecycle-aware runtime harness that improves frozen LLM agents without changing model weights or evaluation environments. Life-Harness evolves from training trajectories by converting recurring interaction failures into reusable interventions across environment contracts, procedural skills, act…

2. Très proche Pas pertinent

   Harness-Zero: Harness Distillation via Agent-as-Harness

   Hao-Ran Ye, Yuxing Lu, Hao-Nan Dong et al. · 2026 · cité 8 fois · Pas encore résumé

   Résumé scientifique

   Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment. Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. We therefore study agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so that its gains survive under a single fixed target harness. The challenge is that the two harnesses differ in action space and av…

3. Très proche Pas pertinent

   JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution

   Gui-Bin Zhang, Leo Lu, Fang-Zhou Xie et al. · 2026 · cité 17 fois · Pas encore résumé

   Résumé scientifique

   Agent capability is not determined by the model alone. The agent harness, encompassing memory management, planning strategy, action protocol, and tool/skill orchestration, can dominate the contribution of the underlying foundation model. Yet harness design remains manual, task-specific, and fundamentally unscalable. We present JIT-Agent, a harness intelligence model trained to synthesize task-adaptive agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. We formalize the agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, and train JIT-Agent to customize harnesses for a given task at hand, repair harnesses for stable and reliab…

4. Très proche Pas pertinent

   Evo-Bench: Can Language Models Improve Agent Harness?

   Lisheng Huang, Chen Yang, Hao Zhou et al. · 2026 · cité 12 fois · Pas encore résumé

   Résumé scientifique

   Large Language Models (LLMs) have driven rapid progress in autonomous agents, yet standard evaluations remain confined to static task solving. An emerging frontier is harness evolution---the agent's capacity to autonomously optimize its own operating harness. However, systematically benchmarking this capability remains challenging, as existing evaluations fail to isolate harness improvements from base model strength, prevent task-specific overfitting, or capture long-horizon iterative research. To address these challenges, we introduce Evo-Bench, the first benchmark designed to evaluate models'intrinsic harness-evolving capabilities across Search, Office, and General agent domains. To rigoro…

5. Très proche Pas pertinent

   Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses

   Tailin Zhou · 2026 · cité 9 fois · Pas encore résumé

   Résumé scientifique

   Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the \emph{harness}---is typically treated as a fixed artifact after deployment. This work studies an alternative where the harness is \emph{task-specific and continuously evolvable}: each task family maintains its own harness, which is hot-swapped across iterations through a fixed task-injection seam and rewritten using environment feedback. We introduce \textbf{Hierarchical Self-Improvement (HSI)}, a framework in which a single frozen LLM $M$ operates across three hierarchical scopes: a task harness $H$ that executes tasks, an evolver that rewrites $…