Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification

2026-08-27 · Jinghan Xu, Yikai Zhang, Aili Chen, Weiyuan Li, Jiaqing Liang, Deqing Yang arxiv

Agent harnesses shape how language-model agents use instructions, tools, and runtime components, but adapting these harnesses requires costly verification. Existing propose-and-verify methods typically score every candidate on a fixed task set, wasting rollouts on unrelated behaviors and allowing aggregate scores to obscure specific regressions. We introduce HarnessLens, a budget-aware framework for automated harness evolution. HarnessLens jointly explores the task space and user-configurable components, derives candidate modifications from execution trajectories, and selectively verifies each candidate on behavior-relevant tasks using an attributable-evidence gate. Across three agent harnesses and four benchmarks, HarnessLens improves average held-out performance by 7.6-13.6% while consuming substantially less evaluation budget than competing baselines. These results demonstrate that behavior-aware verification with explicit attribution enables more reliable and sample-efficient harness evolution under constrained interaction budgets. Our code is available at https://github.com/jhxu5214/HarnessLens.

📄 PDF Abstract BibTeX arXiv:2608.27311

Code (0)

등록된 구현이 없습니다.

LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Search

2026-05-10 · Temoor Tanveer arxiv

LLM-guided evolutionary methods such as AlphaEvolve have proven effective in domains like math, systems research, and algorithmic discovery, but their reliance on frontier models makes each run expensive. We argue this i…

Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails

2026-09-08 · Zhou Yu, Bin Bi, Shiva Kumar Pentyala, Shubham Mehrotra 외 hf

Agent harnesses (the system prompt, tool set, execution hooks, and context-management scaffolding around a model) are a critical determinant of agentic task success. Automated harness evolution can enable smaller models …

DemoEvolve: Overcoming Sparse Feedback in Agentic Harness Evolution with Demonstrations

2026-05-23 · Lirong Che, Yuzhe yang, Peiwen lin, Chuang wang 외 arxiv

Agent harness evolution improves frozen language-model agents by modifying the executable structures around them. We study this paradigm as a form of sample-efficient fast adaptation: instead of updating model weights, a…

Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents

2026-05-28 · Minhua Lin, Juncheng Wu, Zijun Wang, Zhan Shi 외 arxiv

LLM agents are increasingly deployed as systems built around editable external harnesses, including prompts, skills, memories and tools, that shape task execution without changing model parameters. Harness self-evolution…

Instruction Following

Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses

2026-08-09 · Tailin Zhou hf

Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the harness---is typically treated as a fixed artifact after deployment. This…