Understanding Harness Engineering: The Hidden Machine Holding AI Agents Together

A short technical handbook on artificial intelligence agents tells the story from the machine around the model, not the model itself. Signed by techNmak, the work presses one thesis across 43 chapters: a capable language model alone is no agent; the runtime and control rig binding a model to multi-step work is called an agent harness. Anthropic draws the line cleanly: when application code draws most of the path it is a workflow, when the model chooses it is an agent. So the label is not the test; whether model output enters a controlled loop with outside state is.

The smallest loop is written with four objects: durable state, selected context, model output and observation. The loop turns like this: build active context, call the model, return on a final answer, pass action requests through approval and routing, run in the environment, update state with the observation, repeat. The research ancestor is ReAct: interleaving reasoning traces with work actions so outside observations can shape later calls. The ReAct record on arXiv counts as the founding idea of today's rigs; modern harnesses need not keep the traces, they keep the idea: act, observe, update, continue.

The interface is half the decision: ACI and tool design

A model never talks to the computer, it talks to an interface. One rig serves tidy tools that read files by line ranges and apply patches with tests; another serves only a raw shell. Both touch the same repository but pose different puzzles to the model. The SWE-agent study named this the Agent-Computer Interface: never assume a human-built interface suits a model, make the interface a test variable. The SWE-agent record on arXiv gave a historical result for its day; the lasting gift is the question: is the capability present, or presented in usable form?

A tool is not only running code, it is part of model input: name, description, arguments and result shape all enter context. Anthropic thus treats tool design as interface work: clear purpose, single-meaning parameters, collision-free names, high-signal results. Cost splits in two: choice hardship plus observation load. Six overlapping search tools are not richness but a swollen decision node. The rule is crisp: tool count is no one-way measure of agent power. Each added tool opens new behavior while hardening choice; representative work, not taste, should set the balance.

A fault is also an observation back to the model. A blind Error string forces guessing; a branched fault gives the branch, the options and the side-effect-free retry flag. The missing-tool call setting in the OpenAI agent SDK, turning into a model-visible fault, meets Anthropic's actionable-error advice from the other side. The principle holds: a recoverable fault must give enough to choose a better next step without pretending success. The Model Context Protocol enters here: ModelContextProtocol is an interoperability layer between apps and outside capabilities; the Tasks extension defines async work handles with tasks/get and tasks/update calls. But loop scheduling, context choice, approval, containment and grading stay on the harness side; the protocol is not the whole design.

Every model call sees finite context, and the harness picks what enters: instructions, work, history, tool schemas, tool results, docs, plans, memory, sub-agent reports. Stuffing everything hits two walls: capacity is finite and relevance matters; junk fights useful proof for attention. The cure is progressive disclosure: enough to orient, depth discoverable, detail loaded on demand. Capability packs exist for the same reason: open the relevant pack at once instead of every instruction in every prompt. But a 2026 preprint counted 307 confirmed pack faults; packs should be measured, not assumed useful. And summarizing is lossy: C(H) is never H; Anthropic context writings keep durable session apart from active context for this reason.

Approval differs from containment: safety and proof in production

Agent quality shifts when the harness shifts, with no model shift; the April 2026 Claude Code postmortem is the textbook case. Anthropic reported it: with inference healthy, three product-layer changes moved user quality; reasoning settings trimmed then restored, a session-cleaning bug made the model look forgetful, a wording instruction dropped one measure 3 percent then reverted. The lesson is architectural: prompt construction, reasoning settings, context keeping and tool interfaces are part of shipped agent behavior. Raising the model while leaving old harness guesses in place breeds silent loss.

Human approval and containment answer different questions. Approval keeps results under human control; but Anthropic telemetry writes that users approved 93 percent of prompts, and frequent asks breed fatigue. The harder case is a February 2026 controlled leak test: a hostile prompt arriving through the user pulled credentials out in 24 of 25 tries. The defense that held was file and exit borders. The principle in one line: a source that never enters the sandbox cannot be read from it. Credentials should rest outside the sandbox and serve through controlled proxies. The AgentDojo measure draws the same border from the other side: AgentDojo records on GitHub test hidden instructions in tool data with 97 realistic jobs and 629 safety cases.

When an agent says done, that is a claim, not proof. Strong proof lives in outside state: the first repro no longer crashes, regression tests pass, the app ran end to end. The tau-bench design compares final database state against target state and scores repeat success with pass^k; tau-bench records on GitHub lay this order out openly. Stop rules are harness work too: finishing and running dry are not the same; a spent turn budget never proves the work done. The budget vector bounds autonomy: turns, tokens, time, spend, tools, concurrency. The retry rule is sharp: f(f(s)) is never f(s); ticket cutting and card charging multiply under blind retry. RFC 9110 writes the same idea for web requests: automatic retry is a per-tool policy, not a universal strategy.

Measured agent success is a joint function of model, harness, environment, work and grader. Scores move when the environment moves: Anthropic measured a six-point swing from resource shifts on Terminal-Bench 2.0 and advised doubt toward sub-three-point gaps with undocumented method. Google research found architecture effects tied to work across 180 lineups: central coordination gained 80.9 percent on one finance agent while losing 39 to 70 on a sequential plan. In a RetroForge comparison a solo run took 20 minutes at 9 spend units against a full harness at 6 hours and 200; quality rose but spend ran 20 times higher. OpenAI Codex-first work argues agent legibility too: bind rules to mechanism with linters and structural tests, not prose. Every harness part is a current-limit guess; when the model changes, test the guess again. The production list gathers 15 questions: what the model sees, who runs the loop, where work runs, what survives, which rules are mechanical, how faults return, what repeats safely, where proof lives, what proves done, what happens on empty budgets, what follows a break, whether untrusted sights carry orders, how sub-agents coordinate, how the harness is scored. What runs unanswered in demo cannot reason in production.