Harness Engineering: The Model Is a Component, the Harness Is the Product

There's a category of engineering work that has no textbook yet. It isn't prompt engineering — prompts are one input to it. It isn't ML engineering — no training happens. It's the code that sits between a language model and the real world: assembling context, exposing capabilities, enforcing boundaries, capturing state, and deciding what happens when the model produces something wrong.

Call it harness engineering. It's most of the work, and it's the part that survives a model upgrade.

What a harness actually is

A model is a pure function in the way that matters here: tokens in, tokens out, no memory, no side effects, no access to anything. Every capability you associate with an "AI product" — reading a file, calling an API, remembering yesterday, refusing a dangerous action — is code you wrote wrapped around that function.

The harness owns five decisions, and they're the ones that determine whether your product works:

1. What goes into the context window, and in what order
2. What tools exist, and how they're described
3. What the model is allowed to do without asking
4. What persists between turns and sessions
5. What happens on failure — retries, fallbacks, escalation

Notice that none of these are model capabilities. They're all product decisions expressed in code. A GPT-class model with a well-built harness beats a frontier model with a bad one, consistently, on any task that involves more than one step.

The context window is a budget, not a bucket

The most common harness mistake is treating context as storage. You have a big window, so you put things in it: the system prompt, full documentation, the entire conversation, every tool result verbatim. It fits, so it must be fine.

It isn't fine, for three reasons.

Attention is not uniform. Models attend unevenly across long contexts. Information in the middle of a large context is measurably less likely to be used than information at the edges. Dumping 80k tokens in doesn't give the model 80k tokens of working knowledge — it gives it a haystack.

Irrelevant context is actively harmful. It isn't neutral padding. A stale tool result from six steps ago that contradicts current state will get used. A section of documentation about a feature the user didn't ask about will pull the response toward it. Every token you include is a token that can steer the output.

You pay for it every turn. In an agentic loop, context is re-sent on each iteration. A 50k-token context across a 20-step loop is a million tokens of input. Cost and latency both scale with the thing you were treating as free.

The harness discipline: treat every token as something that must earn its place. Retrieve instead of include. Summarize instead of append. Drop instead of keep.

Bad:  [system] + [all docs] + [full history] + [every tool result verbatim]
Good: [system] + [retrieved docs for this step] + [compacted history]
      + [recent results in full, older results summarized]

A useful design constraint: assume the context window is one-tenth of its actual size. Build for that. You'll end up with a system that degrades gracefully instead of one that falls off a cliff at scale.

Context assembly is a pipeline, not a template

Once you accept context as a budget, assembly becomes real engineering. In a mature harness it looks like a pipeline with distinct stages:

Gather — pull the candidates: relevant documents, recent history, tool schemas, user state, environment facts.

Rank — score candidates against the current step. Not the original request — the current step. What mattered at step 1 usually doesn't matter at step 12.

Compact — replace verbose material with dense equivalents. Full file contents become the relevant function plus a note that the rest exists. Ten turns of exploration become three sentences of findings.

Order — place the highest-signal material where the model attends best: near the start and near the end. Instructions early, current task late.

Verify — check the assembled context for contradictions before sending. If step 3 said the config is at path A and step 7 discovered it's at path B, both statements are now in context and one of them will be believed.

That last stage is the one almost nobody builds, and it's the one that prevents the weirdest class of agent failure — the agent that confidently acts on information it already superseded.

Capability boundaries belong in the harness, not the prompt

"Don't delete files without confirming" in a system prompt is a suggestion. It works most of the time, which is worse than not working at all, because it means you'll ship it.

Anything that must be true has to be true in code. The pattern is a permission layer between the model's tool call and actual execution:

type Decision = 'allow' | 'deny' | 'ask';

function authorize(call: ToolCall, ctx: Context): Decision {
  const rule = policy[call.name];
  if (!rule) return 'deny';           // unknown tool: closed by default
  return rule.evaluate(call.args, ctx);
}

Three properties matter here.

Closed by default. An unrecognized tool call is denied, not passed through. The failure mode of an open default is unbounded.

Deterministic. The same call in the same state gets the same decision. If the policy itself is a model call, you've moved the problem, not solved it.

Auditable. Every decision is logged with the reason. When something goes wrong in production you need to answer "why was this allowed" without re-running anything.

This is where I'd push back on a common instinct: people reach for a "safety prompt" because it's fast, and code because it's slow. But a permission layer is a hundred lines and it's the difference between a system you can put in front of a paying customer and a demo. The tradeoff isn't safety versus speed. It's speed now versus a support incident later.

Observability: you cannot debug what you didn't record

An agent that failed is a sequence of decisions, and by the time you hear about it the sequence is gone. Logging the final output tells you nothing — you already know it was wrong.

What you need recorded, per step:

• The exact context sent — not the template, the rendered result
• The raw model output before any parsing
• Which tool was called, with what arguments, and what came back
• The permission decision and its reason
• Token counts and latency per stage

The exact context is the one people skip, because it's large. It's also the only artifact that lets you reproduce a failure. Store it compressed, expire it after thirty days, but store it. "I can't reproduce it" is not a debugging state you want to be in with an agent.

The payoff compounds: once you have step-level traces, your evals write themselves. A production failure becomes a test case by extracting the context at the step where things went wrong.

The harness is your moat, not the model

Here's the strategic argument, and it's the reason I think this is worth taking seriously as a discipline.

Model quality is converging and commoditizing. The gap between the best available model and the good-enough one narrows every release cycle, and the price of the good-enough one collapses. Anything you build that depends on having access to a better model than your competitor has is built on sand.

The harness is different. Your context assembly encodes what you learned about your domain. Your tool set encodes what your users actually need to do. Your permission policy encodes your judgment about risk. Your eval suite encodes every failure you've already survived. None of that transfers when a competitor gets the same model access you have.

A concrete test of whether you've built a harness or just a prompt: swap the model. If changing providers means rewriting your product, you built on the model. If it means changing a config value, running your evals, and adjusting two prompts, you built a harness.

That test is worth running before you need to.

Where to start

If you're building an agent right now and this reads as abstract, three moves in order:

1. Log the rendered context for every model call. Today. Everything else depends on being able to see what the model actually saw.
2. Put a permission function between tool calls and execution, even if every rule currently returns allow. The seam is the point; the rules come later.
3. Set a context budget — an actual number — and enforce it in code. When assembly exceeds it, something has to be dropped, and being forced to choose is what makes you build ranking and compaction.

None of this is glamorous. It's plumbing. But the products that work are the ones where someone did the plumbing, and the demos that never ship are the ones where someone kept tuning the prompt.

"You don't ship a model. You ship everything around it."