An AI agent is two things at once: a model that reasons and a harness that equips it. The harness holds the instructions, the rules it must respect, the memory it may consult, the tools it may call, and the helpers it may delegate to. Google Research calls improving that layer harnessing, and the core claim is simple: agent equals model plus harness. A brilliant model inside a sloppy setup still produces weak work, which is why the same frozen model can score so differently under different setups.
Most teams assemble that setup once and reuse it for every new assignment forever. That works while assignments resemble what the designer imagined, then degrades as fresh, unplanned work arrives. Google calls this setting the open world: tasks nobody anticipated when the harness was drawn up. One practitioner quoted in the video describes the identical pain in his own multi-agent setup, where a failed job almost never demands a new model, only a repaired setup.
Scores First, Mechanism Second
The numbers come from three professional test suites: WorkBuddy-Bench for office, web and coding deliverables, JobBench for end-to-end professional assignments, and LAB, the legal-agent test from Harvey. Across four model-and-harness combinations with GLM-5 and Gemini 3.5 Flash, a single refinement round added up to 11.5 points on WorkBuddy-Bench, 4.5 on JobBench, and 11.8 percentage points of partial-pass rate on LAB. The GitHub repository records the same headline figures in its README, so the video figures match the primary code source.
What makes the result striking is that every gain above came from one improvement round, with model weights untouched throughout. Nobody retrained anything and nobody paid for a new foundation model; only the surrounding files changed. The study on arXiv reports that larger budgets add further gains, with parallel runs up to k equal to five contributing several more points on each suite. That pattern matters because it turns harness work from folklore into a measurable performance axis.
Three Verbs and Eight Files
Each task runs through a contrastive loop with three verbs: propose, probe, compose. A harness editor drafts two candidate setups, one aimed at general weaknesses such as planning or tool use, the other at the current domain. Both candidates attempt the same assignment under matched conditions, and a composer merges only the edits supported by the behavioral contrast. No answer key guides the choice; the AI Primer walkthrough counts four runs per task at the default budget, baseline plus two candidates plus the composed result.
The editable surface covers eight file-mounted parts: the main instructions, the guardrails that mark hard limits, the memory the agent consults, the guidance for tool use, the helper agents for delegation, the capability packs and saved routines it can invoke, and the middleware that observes every turn. The first four shape what the agent knows when it starts; the next two shape how it uses tools and shares work; the last governs what happens inside each turn. Anyone running a modern coding assistant will recognize the same building blocks under different names.
Everything above happens inside a single assignment, which the authors call task-level recursion. Above it sits a second loop, harnessing-level recursion: after each task an experience reflector stores the adopted setup plus the lessons that held up. Those lessons land in a persistent improvement library, a kind of notebook about self-improvement that the next task reads before proposing edits. Crucially, only evidence-backed lessons persist, so the notebook fills with proven moves rather than guesses. The AgentGid review highlights the same split, noting that the default editor behind the headline figures is Claude Opus 4.8, a stronger system than the executors it refines.
Budgets, Limits, and Takeaway
Extra budget can be spent two ways. Parallel recursion tests several setups at once and builds combined successors from everything learned, exploring wider. Sequential recursion takes the best setup and refines it again, digging deeper. Both directions improve with budget. Execution currently runs through OpenCode by default or Codex as an option, which grounds the work in tools teams already use rather than a lab-only rig.
The honest costs come next. Every probe is a full run of the agent on the assignment, so testing six setups means paying for six runs plus the editor sessions, which is why the official guide advises starting with a small task set. The release is version 0.1.0, explicitly not an officially supported Google product, and it expects a Linux host with Python 3.12 rather than a one-click app. The TheAIFeed report asks the right follow-up: with gains established, the practical question becomes execution budget. Executor network access stays unrestricted and the Codex sandbox runs disabled, so benchmark runs belong inside an isolated sandbox.
Strip away the tooling and the durable lesson is portable: fix the setup before blaming the brain. Instructions, rules, memory routines and capability packs are plain prose anyone can edit, so the same discipline helps whatever agent you run today. My own yardstick for such frameworks is whether the improvement loop compounds without human babysitting, and the library mechanism is the part that could. Weak executors gained the most in the reported figures, which suggests the first wins await the messiest setups, not the strongest models.
| Loop | What changes |
|---|---|
| Propose-Probe-Compose | Two candidates run, composer keeps supported edits |
| Improvement library | Adopted setups plus proven lessons guide next task |
| Single-round gains | Up to 11.5 points with weights frozen throughout |
Key moments
AI commentary
"I read this as a shift in where progress comes from: not a larger brain, but a tidier workshop around it. The idea that each job can sharpen the next one is what sold me, and the evidence-backed notebook is the part I would copy first. TheAIFeed report frames the same release as a research preview, which matches my take: exciting mechanism, early software."
AI assessment
The strongest counter-argument starts with cost and randomness. Extra runs multiply token bills, behavioral comparison can misfire when executions vary for unrelated reasons, and the study itself documents selections that preferred weaker candidates. Teams should therefore track spend per improvement round and keep a human in the acceptance path.
A second caution concerns grading. Refinement needs no answer key, yet the published scores were judged by Claude systems against rubrics, so the loop that needs no judge is validated by judges. Independent replication on private tasks, with network-isolated executors, is the missing confirmation I want before recommending production use.
Practically, start without the framework at all. Open the instruction files of your current agent, tighten one vague rule per week, and log which change altered behavior. If that habit sticks, the Harness Squared demo, which runs both recursion modes without model credentials, is a cheap next laboratory. Treat benchmark deltas as directional, not promises.
Setup honesty matters too. Model-backed runs expect a Linux host with Python 3.12, Node.js, a Vertex AI project, and per-test dependencies such as Docker or Pandoc, while the deterministic demo stays free of credentials. The code ships under Apache 2.0 as research software at version 0.1.0, so expect rough edges and plan sandboxing from day one.
Sources
6 links; no other published story cites them. Stories sharing a link do not confirm each other; a source's origin is not inferred from how often it is cited.
harness · frozen model · agent loop