Skip to content

Harness Patterns

The harness is the scaffolding around the agent — prompts, tools, context policies, sandboxes, and feedback loops. It is a first-class engineering artefact that requires continuous refinement, not a disposable wrapper. An operational test for what actually counts as a harness (as distinct from a framework, SDK, or orchestrator) — validated against Claude Code, Codex CLI, Aider, Cline, OpenHands, and SWE-agent — is given in "What makes a harness a harness" (arXiv, Jun 2026). Anthropic's own engineering blog states a shorter, compatible definition in passing while covering agent evals rather than harness design directly: "the system that enables a model to act as an agent: it processes inputs, orchestrates tool calls, and returns results." (Anthropic, "Demystifying evals for AI agents", Sep 2026.)

OpenAI's own primary-source definition and case study. Where Anthropic's definition above is a passing aside in an evals post, OpenAI's own engineering blog is a dedicated first-party account: five months, 0 lines of human-written code, ~1M LOC, ~1,500 merged PRs across 3→7 engineers (3.5 PRs/engineer/day, throughput increasing as the team grew), with AGENTS.md deliberately kept to ~100 lines as a "table of contents" pointing into a structured docs/ system-of-record rather than a monolithic instruction file. A recurring cleanup that once cost a dedicated 20%-of-week "AI slop" day was replaced by codifying "golden principles" into linters and a scheduled Codex-driven cleanup pass. (OpenAI, "Harness engineering: leveraging Codex in an agent-first world", Feb 2026; direct fetch 403s — captured via Wayback.)

A larger follow-on source-code study extends the validated-harness test above to eleven production coding harnesses (Claude Code, Codex CLI, Gemini CLI, Mistral Vibe, OpenHands, Aider, Mini-SWE-Agent, Hermes, Pi, OpenCode, OpenClaw) plus Omnigent as a meta-harness case — a source-code-level anatomy/architecture/evolution comparison rather than the definitional test above. (arXiv 2609.00006, "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents", Sep 2026.)

Its headline finding bears on who owns the loop: across roughly four million lines "no agent runtime imports a general-purpose agentic framework" (LangChain, LangGraph, AutoGen, CrewAI and a dozen others were checked) and none retrieves code with vector embeddings; every loop is hand-rolled, which the authors put down to "debuggability and prompt transparency" outweighing framework reuse when the agent mutates real code. It is a source study of coding harnesses, with Claude Code read from "a circulated source snapshot". A practitioner write-up (@wasowskijarek, 19 Oct 2026) reframes the choice as who owns the loop and where state must survive a crash: its framing, not the paper's.

The commercial stakes are not abstract. Reporting on Cursor's SDK launch and its compute deal with xAI, The New Stack frames the shift as: "models are becoming commodities, and the harness is becoming the product." The article's own headline claim — "Cursor's $60 billion bet" — is not Cursor's valuation; it is a secondhand report (Bloomberg/TechCrunch, cited within the article) that SpaceX "will either pay Cursor $10 billion for the companies' work together or acquire Cursor outright for $60 billion later this year." Read as evidence for a strategic thesis (a compute/chip vendor valuing the harness and its developer relationship over the underlying model weights), not as a confirmed valuation figure. (The New Stack, "Cursor's $60 billion bet is on the harness, not the model", May 2026.)

Harness vs. Loop — Two Architectural Layers

Layer Scope Analogy
Harness Single-agent safety wrapper — prompts, tools, context policies, sandboxes Equipment
Loop Multi-agent orchestration + scheduler that governs multiple harness cycles Factory control plane

The harness is prerequisite infrastructure; the loop is the control plane above it. A well-designed loop depends on well-designed harnesses — but the loop's job is coordination and termination, not execution.

The Harness as an Org-Level Artifact

At scale the harness stops being a per-developer convenience and becomes the primary organizational artifact — the shared substrate that defines how every agent in a company behaves, what it may touch, and how its work is verified.

"It is an org-level harness. The difference will become clearer over time." (@karpathy on Claude Tag, Jun 2026)

The implication: harness design decisions (permission posture, verification gates, credential handling) are no longer local choices — they propagate to every loop the org runs. This is why the harness, not the model, is the leverage point ("the harness now matters more than the model" — see The Paradigm Shift).

Origin story. Anthropic's own account of Claude Code's beginnings (internal codename "clide") traces "harness design" back to 2022-2023 work on giving a model a persistent shell and a bash tool — before Claude Code existed as a product, the prototype was already fanning out 100 Haiku subagents in parallel, and the app itself was, in its builders' words, "tool definitions in a loop and a simple REPL UI." The org-level-artifact framing above isn't a new idea layered on top of the tool — it's the same concern the original builders were solving for from day one. (Anthropic, "The Making of Claude Code", Jul 2026.) The org-level harness is realised concretely as per-thread/per-channel agent instances with their own memory and permissions — see Claude Tag — and governed across many loops via Fleet Engineering.

Brownfield Agentic Engineering

Most of this doc's harness design guidance assumes a codebase the agent is free to restructure. Brownfield Agentic Engineering is the harness-design counterpart for legacy codebases, where the constraint is unfamiliar, high-risk code the agent must not break: risk-zone tiers gate agent autonomy by code-area risk, durable comprehension-memo artifacts capture what an agent learned about a legacy module so the next session doesn't re-derive it, and mandatory characterization tests are written before any change is allowed — the agent must first prove it understands current behavior, not just intended behavior. (Addy Osmani, "Brownfield Agentic Engineering", Sep 2026.)

Two controlled studies quantify how much this constraint matters in practice. On SWE Refactor Bench, only 28 of 520 agent refactoring runs passed migration audit, behavioral tests, and independent verification (arXiv 2608.23564, Aug 2026). A VB6-to-C# migration study found 92% behavioral equivalence on simple migrated features vs. only 47% on complex ones (arXiv 2608.28972, Aug 2026) — the same risk-tiering brownfield framework argues for, measured directly.

The quantified version of the thesis: LangChain reports moving their coding agent "from Top 30 to Top 5 on Terminal Bench 2.0 by only changing the harness" — same model (Opus 4.6 in Claude Code), harness-only changes. This is the hardest evidence to date that harness design, not model swap, is the accessible leverage: a mid-pack agent became top-5 with the model held fixed. (LangChain, "The Anatomy of an Agent Harness", Mar 2026.)

Two benchmarks add further quantified weight to the same thesis. StaminaBench (stress-testing coding agents over 100+ interaction turns) finds that for a strong model, harness quality alone creates up to a 6x performance gap between that model's best and worst harness, and that feedback loops improve results by up to 12x over single-shot attempts — the harness, not the base model, dominates sustained-task performance. (arXiv 2606.19613, Jun 2026.) Claw-SWE-Bench (evaluating OpenClaw-style harnesses on coding tasks) found the same backbone model scores only 19.1% with a minimal adapter versus 73.4% with a full adapter — a 4x swing from harness completeness alone, with the model held fixed. (arXiv 2606.12344, Jun 2026.) A smaller independent pilot corroborates the pattern at the opposite end of the scale: comparing Claude Code against the Pi harness on 19 Terminal-Bench tasks with the model pinned identical, both harnesses reached the same 87.7% task success rate, but Claude Code used 5.2x more tokens (geometric mean) to get there — after the author corrected an initial pass that had shown a contaminated 7-14x gap. Read together with the StaminaBench/Claw-SWE-Bench results above: harness choice can move either the success rate or the cost at matched success, and a harness comparison that only measures one of the two will miss which lever actually moved. (nmlemus/harness-token-efficiency, Sep 2026.)

A fourth, independently run comparison finds the widest gap yet. Running Aider, Claude Code, and OpenClaw on the same underlying model, three benchmarks measured 3,500 to 292,000 tokens-per-task across identical tasks — a 70-fold spread from harness choice alone. The dominant driver is a "startup tax": harnesses that resend the full system prompt and tool-description catalogue on every turn rather than caching it, a variable that alone predicts tokens-per-task at R²=0.99 across the tested harnesses. Read together with the StaminaBench/Claw-SWE-Bench/harness-token-efficiency results above, harness-driven cost variance is now corroborated by four independent measurements, none sharing an author. (The New Stack, "Aider, Claude Code, and OpenClaw ran an identical model. Token use varied 70-fold.", Sep 2026.)

A fifth, purpose-built research benchmark for the same variable — not to be confused with the harness-bench conformance suite below (omnigent's pass/fail capability test). This Harness-Bench is a diagnostic benchmark: 106 sandboxed tasks across multiple model backends under matched budgets, measuring completion, process quality, and efficiency as a function of harness configuration alone. Across 5,194 execution trajectories it finds "substantial variation" across model-harness pairings and recurring execution-alignment failures (plausible reasoning decoupled from tool feedback or workspace state) — converging on the same recommendation as this section: report agent capability at the model-harness configuration level, not the base model alone. (arXiv 2605.27922, "Harness-Bench: Measuring Harness Effects across Models", May 2026.)

A sixth measurement comes from a competing open-source harness benchmarking itself against Claude Code and Codex. AWS's Strands Harness, an open-source general-purpose agent harness (loop, tools, context management, session handling, delegation) built on the Strands Agents SDK, reports 28% lower token cost than Claude Code and Codex across six benchmarks, crediting its context-compaction defaults — a secondary write-up reported the gap as 45% cheaper instead, so the figure here is the primary one; treat the discrepancy as unresolved rather than picking a number. (Strands Agents, "Introducing Strands Harness: Frontier Performance With 28% Lower Token Cost", Sep 2026; SDK: strands-agents/harness-sdk.)

A seventh, model-invariant data point: a "Writer Agent Harness" redesign cut cost/task 41%, wall-clock 44%, and tokens 38% with no quality loss, consistently across six different foundation models — evidence the harness-design gain generalizes across models rather than being a tuning artifact of one. (arXiv 2607.06906, "The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI", 2026.)

An eighth measurement holds the model constant against a vendor's own managed offering rather than a third-party harness: a self-hosted "True Forge" harness (True Foundry) used 62% fewer tokens than Claude's own managed agent platform at equal accuracy with Opus held constant — the harness-cost gap shows up even against a vendor's own managed platform, not only between third-party harnesses. (MindStudio, "Why Your AI Agent's Harness Matters More Than the Model for Cost", 2026.)

A second, independent LangChain data point with a different model corroborates the Top-30→Top-5 result above: Terminal-Bench 2.0 score moved 52.8%→66.5% (+13.7 points) with GPT-5.2-Codex held fixed, harness-only. (via explainx.ai, "Agent Harness Engineering: Terminal-Bench, LangChain Lessons", attributing LangChain's Feb 2026 blog post.)

A model score is not an agent score — and a benchmark methodology to isolate why. All the measurements above compare a harness's aggregate cost or success rate; a proposed methodology goes one level deeper, decomposing "agent score" into 11 harness components (tool adapter, retry policy, context policy, orchestration loop, stopping rule, and six more) any one of which can fail independent of the model. It proposes running a "pinned control loop" — a fixed reference harness — against a "submitted configuration" to isolate what changed. The accompanying benchmark's HDS6 metric scores Tools, Repair, Alternatives, Coherence, Evidence, and Scope independently of final-task success, so a harness can be diagnosed on which component degraded rather than only that the score dropped — a finer-grained instrument than this section's cost/success comparisons, though not yet cross-validated against any of them on the same tasks. (@akshay_pachaar; arXiv 2608.11341, "Apodex Discovery: Reality Benchmarks and Environments for Evaluating Discoverative AI", Aug 2026.)

The ranking does not carry across models. In the Apodex paper's own table (seven diagnostic environments, mean score) the same four model-agnostic harnesses reorder: on Opus-4.8 ApodexHarness leads (0.611) and A-Evolve is third (0.569); on GPT-5.6-sol A-Evolve leads (0.648) and ApodexHarness is second (0.592). The authors conclude "No single harness is universally optimal"; they also built ApodexHarness, and the post above was written with Apodex. The Hugging Face / Liquid AI multi-harness RL guide (Sep 2026) cites Joel Niklaus's SWE-bench Pro runs of GLM-5.2 at 23% in one harness and 52% in another, then trains a 2.6B model with RL inside four unmodified harnesses (Claude Code, Codex, OpenCode, Mini-SWE-Agent): held-out pass@1 rose from 42% to 54% averaged over the four, while OpenCode-only training reached 52.3% ("within the noise") but kept its gain mostly in OpenCode (34% to 58%). One seed per run and unequal data exposure, so the guide calls the comparison "observational". For a loop that can swap model or harness, a score belongs to the pair; re-measure after either changes.

Two Settings Tripled a Benchmark Score — and the Vendor Didn't Sell the Harness

The sharpest evidence yet for "the harness matters more than the model" came from a vendor demonstrating it about itself. OpenAI found that the "official" ARC-AGI-3 harness was erasing the model's private reasoning and truncating its history every move — enabling reasoning-retention plus context compaction tripled the score (13.3%→38.3%) at 1/6th the tokens, with the model held completely fixed. (OpenAI, "How enabling two settings tripled our scores on the ARC-AGI-3 benchmark", Jul 2026.) The compaction mechanism itself — opaque, loss-aware, server-side compression of prior reasoning/tool calls — is documented as a first-class API primitive precisely because naive truncation this badly starves a long-running loop of exactly the context it needs to avoid repeating dead ends. (OpenAI, Compaction developer guide, Jul 2026.)

The same pattern recurred at the other end of 2026, this time without the vendor's own admission. GPT-6 Astra's headline 98.6% ARC-AGI-3 score came from an undisclosed evaluation harness/system wrapped around the model — a harness OpenAI does not sell to developers — rather than the raw model under conditions anyone could reproduce. (The New Stack, "OpenAI will sell you Astra, but not the system that scored 98.6%"; The New Stack, "GPT-6 Astra's score of 98.6% looked like AGI. Then researchers read the fine print.", Sep 2026.) Read the two events together and the lesson sharpens past "the harness matters": a vendor that understands this thesis has an incentive to under-disclose which parts of a benchmark score are harness rather than model — see Benchmark and Eval Integrity for the verification-side treatment of the same disclosure gap. ARC Prize's own reference harness isolates exactly this variable as an experiment rather than an incident: swapping a "Standard" harness for a "Provider Adapter" harness around the same model moved one score from 62.7% to 99.9%, a ~37-point swing from harness alone. (ARC Prize Foundation, "OpenAI's GPT-6 Astra on ARC-AGI-3", Sep 2026; open-source harness: arcprize/arc-agi-3-benchmarking.)

A counterpoint from the same benchmark family: harness-light also wins. ARC-AGI-3's first Milestone Prize went to a team whose stated philosophy was the opposite of building a bigger harness — "keep the harness lightweight and generic and let the model drive," reporting that hand-crafted tools hurt performance relative to letting the model improvise. Read against the Astra/Provider-Adapter result above, the two are not actually in tension: both show the harness moving the score, just in opposite directions for different failure modes — Astra's harness was compensating for missing capability the model didn't have, this team's lightweight harness was removing scaffolding that was constraining a model that didn't need it. The design implication is the same either way — measure which failure mode you have before choosing which way to move the harness. (ARC Prize Foundation, "ARC-AGI-3 Milestone Prize #1", Sep 2026.)

Persistent "megathreads" as the harness-level fix for long-running work. A companion practice guide from the same period generalizes the reasoning-retention lesson into a standing recommendation: keep one long-lived agent thread alive and deliberately compact it on purpose (rather than starting fresh each task), decompose goals into independently verifiable steps, and decide upfront which steps get full delegation versus which keep a human checkpoint. This is the harness-design counterpart to Long-Running Agents' session-recovery patterns — persistence by keeping the thread alive, rather than by reconstructing state after a reset. (OpenAI, "Codex-maxxing for long-running work", Jun 2026.)

Harness Conformance Testing (harness-bench)

If the harness is the leverage point, it needs its own tests — not just the code it produces. A harness conformance suite validates whether a given harness meets a standard capability contract (does it enforce stop conditions, isolate agents, gate tools, recover state?), turning "is this harness production-grade?" into a benchmark rather than a judgement call. This is a distinct evaluation primitive from output verification (docs/04): output verification asks "is the work correct?"; conformance testing asks "is the harness capable?" (omnigent-ai/omnigent harness-bench, Jul 2026.)

Self-correcting capability tables. A conformance bench is only as trustworthy as the capability table it checks against — and that table can itself drift out of sync with reality (e.g. an adapter wrongly declared "streaming-capable" for a harness that dropped streaming support). The fix is structural: when a conformance run finds a declared capability doesn't hold, the bench corrects the source model (the capability declaration) rather than papering over it with a special-cased test — keeping the declared contract and the bench itself from silently diverging over time. Supporting this, the bench runs against three transport drivers (in-process SDK, full server, native TUI) so a capability gap specific to one transport doesn't hide behind a pass on another. (omnigent-ai/omnigent harness-bench, Jul 2026.)

Cross-vendor observable policy denials. A conformance suite is only useful if a denial is actually visible to whatever is testing it. harness-bench added a PolicyDeniedEvent so a native harness's (Claude Code, Codex, etc.) policy denial can surface as an observable stream event rather than a silent no-op — letting the same conformance probe verify deny-behavior consistently across vendors instead of trusting each harness's own logs. The mechanism landed in #2096 (Jul 8) but wasn't enforced live until the native policy hook was wired up the next day in #2171 (Jul 9), after which the deny path was confirmed working end to end. (omnigent-ai/omnigent harness-bench #2096, #2171, Jul 2026.)

Schema-Level Conformance (Temper)

harness-bench tests behavior (does the harness enforce stop conditions, gate tools, recover state at runtime?). A complementary, cheaper check is schema-level conformance: does the .claude/ directory itself — skills, rules, agents, hooks — match a declared contract, before anything even runs? Temper implements this as a compiler-like pipeline rather than a linter: install scans the whole .claude/ directory into one typed model (converting each discovered artifact into a typed member module), emit deterministically compiles author-declared requirements into a lock file (regenerated twice to self-verify determinism), and check gates the actual files against that lock in CI. The distinction from harness-bench: this catches drift in the harness's declared shape (a skill file that no longer matches its own schema) before a conformance run ever needs to exercise it at runtime. (duct-tape-and-markdown/temper, Jul 2026.)

Task-Shaped DAG Orchestration

An alternative to a fixed N-agent pipeline (two-part, three-agent, five-wave): compose a task-shaped DAG per issue, sized to the issue's actual complexity rather than a fixed role count. A simple fix becomes a single-node DAG; a complex feature fans out into parallel research and implementation legs. Distinguishing mechanisms:

  • Fail-closed exit gates: every node runs an act → check → close cycle; a post-dominance gate ensures a code-reviewer node evaluates every code-producing node reachable in the graph (no path bypasses review by construction), and a security-reviewer gate is inserted specifically for sensitive changes.
  • Classifier-gated parallelism: before claiming parallel work, a pre-claim check (file overlap, shared dependencies, shared infrastructure) marks candidate work green / yellow / red / blocked — parallelism is only attempted where the classifier has already ruled out collision, rather than discovered after the fact (contrast with Scope-Verified Parallelism, which catches collisions at the point of write instead of before dispatch — the two are complementary layers, not substitutes).
  • Scripted octopus merge, agent-gated conflict resolution: write-fan-outs proven disjoint still run in isolated per-node worktrees by default and are merged mechanically by a script (a git octopus merge, no agent involved); only when that merge hits a real textual conflict is a dedicated synthesizer agent dispatched to resolve it by intent — parallel execution never merges itself, but the agent is reserved for the conflict path, not the disjoint one.
  • Bounded review-fix loop: capped at a maximum of 5 iterations with mechanical (not self-assessed) verdicts, preventing the infinite fix loop pattern.

Durable state (workflow-state.md) records phase, step, pending gates, and per-node evidence, so a session resumes mid-workflow across a context reset rather than restarting the DAG. (KaolaBrother/Kaola-Workflow, Jul 2026.)

Two further cost-shaping mechanisms for the same gated-DAG shape:

  • Gate-first fan-out: a cheap, single dedicated reviewer runs alone first; the expensive N-lens parallel fan-out only runs to confirm a clean verdict from that dedicated pass. A failing round never pays the fan-out cost — parallelism is spent only where it can change the outcome, the same principle as Classifier-Gated Parallelism above, applied to review depth instead of write-collision risk.
  • Capture-once, inject-many: one git diff is captured per review round and injected into every reviewer's prompt, instead of letting each of several concurrent reviewers independently re-run git diff and re-read the same files — measured at ~7.8x redundant reads avoided in the source repo's own accounting. External corroboration for the bounded-loop cap itself: LangGraph ships the identical guard as recursion_limit (default 25), raising GraphRecursionError when a graph loop never converges. (max9159/system-design, Sep 2026.)

Self-Improving Harnesses

If the harness is the leverage point — and it can be tested (harness conformance) — the next step is a harness that optimizes itself. The 2026 research converges on a claim the rest of this doc only implies: the harness is not a fixed, human-authored artifact but an optimization target with measurable returns, and the optimizer can be the agent working on its own execution traces — no stronger external model or human engineer required.

This is the disciplined answer to an under-performing loop (see Failure Patterns): instead of hand-tuning prompts, mine the traces for the harness change that fixes the failure class. It is distinct from Learned Orchestration, which trains a model to orchestrate — here the model stays fixed and the harness config (tools, middleware, memory, control flow) is what evolves.

A first-party production instance: Warp's inner/outer skill split. An inner (base) skill does the actual per-task work (e.g. PR review); an outer "improver" skill runs as a scheduled observer, not per-task — it pulls accumulated human feedback, compares what the base skill suggested against how humans actually responded, and proposes a small, focused edit to the base skill. Critically, the update path is not autonomous: "these updates, which are reviewable, approvable, and mergeable, can flow through a normal PR/code-review workflow; once merged, the next run of the inner skill inherits the improvement." Warp runs this same pattern across separate spec-writing, review, and triage agents, each carrying its own self-improvement loop — a concrete, shipped answer to "who improves the harness" that keeps a human gate on every change, unlike the fully autonomous research systems below. (Anthropic, "How Warp builds self-improving agents on Claude", Aug 2026.)

Warp's stated methodology precedes the skill split itself: "Is your domain verifiable? Build the verification harness first, then let the agent tune against it: generate a reference corpus, compare output to reference, fix, repeat." For non-verifiable domains, the fallback is deterministic evals against golden outputs where they exist, restricting human feedback to domain experts rather than opening evaluation broadly — harness-before-tuning as a sequencing rule, not just an architecture choice.

Self-Harness — weakness mining → propose → validate. A three-stage self-improvement loop that needs no external engineer:

  1. Weakness Mining — extract model-specific failure patterns from execution traces.
  2. Harness Proposal — generate targeted, minimal harness edits addressing those failures (not generic prompt instructions).
  3. Proposal Validation — regression-test each candidate before acceptance. The validate stage is itself a verification gate: a proposed harness edit is unverified until a held-out run confirms it.

Reported Terminal-Bench-2.0 gains, model held fixed: MiniMax M2.5 40.5%→61.9%, Qwen3.5-35B-A3B 23.8%→38.1%, GLM-5 42.9%→57.1% (up to +21.4pp absolute). The point: model-specific weaknesses become concrete, executable harness changes rather than more prose. (Figures are from v1, the version live at capture; v2/v3 — Aug 2026 — broadened the study to three benchmarks and nine model–benchmark pairs and restate the headline as relative gains of up to 132%.) (arXiv 2606.09498v1, v1, Jun 2026.)

AHE — observability-driven evolution with verified prediction contracts. Agentic Harness Engineering makes the harness auto-evolvable by building on three observability pillars:

Pillar What it exposes
Component observability Each editable harness element is file-level, explicit, and reversible
Experience observability Raw trajectories are converted into a layered evidence corpus
Decision observability Every edit ships a self-declared prediction, later verified against the next round's task outcomes

Decision observability is the key discipline: each harness edit is a falsifiable contract (a prediction a later run confirms or refutes) — the maker/checker principle applied to the harness's own changes. Results: Terminal-Bench 2 Pass@1 69.7%→77.0% over 10 iterations, beating the human-designed Codex-CLI baseline (71.9%); top SWE-bench-verified aggregate with 12% fewer tokens than the seed harness; cross-family transfer +5.1 to +10.1pp. Ablation: the gains come from tools, middleware, and memory components — not system prompts — which sharpens where harness effort actually pays. (arXiv 2604.25850, Apr 2026.)

HarnessX — a substitution algebra over typed primitives. Frames runtime components as typed harness primitives over a substitution algebra (any primitive can be swapped for an equivalent), with an evolution engine (AEGIS) that refines prompts, tools, memory, and control flow from execution traces. Reports +14.5% average across five benchmarks (ALFWorld, GAIA, WebShop, and two others), up to +44.0%. (Cobus Greyling / arXiv 2606.14249, Jul 2026.)

SEAGym — evaluation environments for self-evolving harnesses. A dedicated eval environment for self-improving harnesses surfaces two risks the three approaches above share: useful intermediate harness snapshots can collapse in later iterations (the optimizer overfits to recent traces and regresses on cases it previously handled), and source diversity — how varied the training traces are — materially affects harness reliability after evolution. Practical implication: keep a fixed regression suite of earlier snapshots' solved cases and re-check it every iteration, not just the newest failures. (arXiv 2606.17546, Jun 2026.)

APEX — co-evolving harness, principles, and workflow together. Rather than evolving the harness alone, a three-layer framework simultaneously co-evolves the harness configuration, the stated principles guiding it, and the workflow topology, raising a composite "Health Score" from 0.300 to 0.570 — evidence that harness-only evolution (the three approaches above) may be leaving gains on the table by holding principles and topology fixed. (arXiv 2606.15363, Jun 2026.)

Darwin Mode — train/eval-disjoint held-out gating (closing the SEAGym collapse risk). SEAGym (above) warns that self-evolving harnesses can overfit to their own recent traces and regress on cases they previously solved. A concrete gating mechanism closes this: the optimizer mutates its own config, sandbox-tests each candidate mutation, and keeps only mutations that measurably improve performance on a held-out benchmark set strictly disjoint from the traces used to generate the mutation (e.g. SWE-bench Lite, LiveCodeBench). Train/eval disjointness — not just "a regression suite exists" — is what prevents the optimizer from mutating toward whatever the held-out set happens to reward. A companion model router learns, from fleet-wide eval logs, the cheapest sufficient model per task rather than always routing to the strongest — a fleet-maturity-relevant detail alongside the harness-evolution one. (ruvnet/metaharness, Jul 2026.)

SoL-Pi — an auto-research loop aimed at efficiency, with the gates fixed in advance. In NVIDIA/MIT's SoL-Pi (17 Sep 2026; code: NVlabs/SoL-Pi) an AI researcher reads the base harness's own traces and proposes changes; one survives only if every capability metric stays inside a predeclared tolerance and an efficiency metric improves. The paper keeps these "strictly isolated from the optimizing agent" and says "a failed validation rejects the candidate without triggering further optimization"; of the 51 public EdgeBench tasks, 11 gate acceptance and 40 are the final test. 152 proposed directions over 535 environments left four mechanisms (costs, context). Authors' results, unreplicated; transfer to a second model is called "preliminary evidence". Reusable rule: the optimizer cannot edit the acceptance criteria.

Mechanized rules beat prose guidance, quantified. A self-improving harness that converts confirmed tool-call failures into durable, vote-weighted lessons (promoted into an enforced hook once repeat evidence crosses a weight threshold) cites — as an unsourced "practitioner consensus," not a measurement of its own deployment — ~100% compliance for mechanized rules versus ~70-90% for the equivalent prose guidance in CLAUDE.md. This is a directly quantified version of the "encode learnings as rules, not prose" principle already implicit in Experience Encoding — a rule the harness enforces is followed far more reliably than a rule the model is merely told about. (Aditya-Nagariya/harness-forge, Jul 2026.)

The shared shape: all three are outer loops whose product is a better harness, each gated by a verifier (regression run / prediction contract / benchmark). They validate the harness-conformance idea from the other direction — not "does this harness pass a fixed capability bar?" but "can the harness raise its own bar and prove it?" Keep the edits minimal and reversible (component observability) and gate every proposed edit on a held-out run — an un-validated harness edit is Verifier Theater at the meta level.

Fix the process, not the code (Bun case study). The self-improving-harness research above assumes an automated optimizer; the same discipline works manually and is easy to skip under time pressure. During Bun's 11-day, 64-parallel-instance Zig→Rust rewrite, Claude instances began repeatedly running dangerous git commands (git stash, git reset) mid-task. The response was not to intervene per-instance or hand-fix the resulting damage — it was to edit the workflow instructions once, globally, so the fix applied to every future instance rather than the one caught in the act. The author's own framing: "fixing the process that generates the code instead of hand-fixing the code." This is the manual, single-engineer version of what Self-Harness and AHE do with an automated weakness-mining step — the failure class, not the individual failure, is what gets patched. (Bun, "Bun, in Rust", Jul 2026.)

JIT-Agent — harness intelligence as a trainable dimension, orthogonal to model scale. Rather than evolving one fixed harness, a trained model synthesizes a task-adaptive harness on-the-fly for whatever off-the-shelf LLM it is paired with — enabling a smaller model with a JIT-synthesized harness to surpass a larger model running the generic default. This reframes "harness intelligence" as a capability you can train for, not just a one-time engineering artifact. (arXiv 2608.25593, "JIT-Agent", Aug 2026.)

HarnessLens — cheaper verification for harness evolution. The self-improving approaches above all gate mutations on some form of re-evaluation; HarnessLens replaces full re-evaluation with behavior-aware verification, improving held-out performance by 7.6–13.6% while using substantially less evaluation budget per candidate — the efficiency counterpart to Darwin Mode's train/eval-disjoint correctness gate above: cheaper does not mean less rigorous if the verification still catches regressions. (arXiv 2608.27311, "Verify Smarter, Evolve Further", Aug 2026.)

EvoUndo — recoverability as its own gate, alongside correctness. A framework that checks whether a model's self-modification to its own harness is recoverable before accepting it, independent of whether the mutation measurably improves performance. Oracle analysis found 197 capability-improving mutations that failed a recoverability check; an extended recovery calculus recovered 191 of the 197 — meaning most "good but risky" mutations can be made safe rather than simply rejected outright. This closes a gap none of the approaches above name explicitly: Darwin Mode gates on whether a mutation helps; EvoUndo gates on whether you can undo it if it doesn't. (arXiv 2608.28363, "EvoUndo", Aug 2026.)

HarnessDev — can a model build its own execution infrastructure from a minimal seed? Tests whether LLMs can construct and refine their own agent harness rather than only tuning an existing one. Generated harnesses lag human-authored references on code and search tasks, but match or exceed them on writing and ML-experimentation tasks — a domain-dependent answer, not a uniform yes or no, to the question the self-improving-harness cluster above otherwise treats as settled. (arXiv 2609.01437, "HarnessDev", Sep 2026.)

Meta-Harness (Stanford IRIS Lab) — an agentic proposer for automated harness optimization. Name collision only — distinct from Omnigent's product feature of the same name below (Meta-Harness: 3-Tier Policy Hierarchy). Trains an agentic proposer that searches over harness code variants using prior candidates' code, scores, and execution traces as context — reusing evolutionary history as in-context evidence rather than starting each mutation from scratch. Shows gains on text classification, math reasoning, and TerminalBench-2. (arXiv 2603.28052, 2026.)

RobustSGPO — search-space control for harness evolution. Rather than gating mutations after the fact (Darwin Mode, EvoUndo above), this constrains where evolution is allowed to search in the first place: periodic permission scheduling raised held-out completion from 60% to 80%. (arXiv 2609.09646, Sep 2026.)

PRISM — failure clustering with Pareto-routed repairs. Clusters tool-agent harness failures and routes each cluster's repair to the surface that fits (prompt edit / middleware / joint edit) via Pareto search, yielding 10-15 point held-out lifts — a repair-routing mechanism distinct from APEX's co-evolution above. (arXiv 2609.05736, "Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses", Sep 2026.)

A second, complementary ablation isolating verification as a harness component specifically (distinct from AHE's tools/middleware/memory ablation above): testing planning guidance, execution organization, and completion verification separately, one paper finds its verifier component rejects 61% of invalid episodes while retaining 17% of correct ones — a precision/recall trade-off the AHE ablation doesn't surface. (arXiv 2609.20474, "How Do Agent Harnesses Create Value?", Sep 2026.)

Harness-of-Harness — a multi-day case study. Coding agents iteratively improve their own software via planning-coding-testing loops sustained over multiple days, reporting a 52.25% average relative gain across three benchmarks and, as a demonstration, autonomously building a working FPS game over 70+ iterations. This is the sharpest available evidence that the self-improving-harness pattern holds up over genuinely long horizons, not just a handful of iterations in a benchmark run — see Long-Running Agents for the session-continuity mechanics a run this long depends on. (arXiv 2609.01481, "Harness-of-Harness", Sep 2026.)

Four late-September arXiv additions extend the cluster along three different axes. Distillation: Harness-Zero treats the agent itself as the harness and distills a compact policy out of it via Agent-as-Harness — moving the self-improvement loop's target from prompts/middleware (AHE, RobustSGPO, PRISM above) to the agent's own weights. (arXiv 2609.24974, Sep 2026.) Regularization: RRSI adds explicit regularization to the recursive-self-improvement loop itself, addressing the snapshot-collapse risk this cluster already tracks via train/eval-disjoint held-out gating — a second, independent mechanism for the same failure mode. (arXiv 2609.24972, Sep 2026.) Decentralization: Agensh removes the central orchestrator entirely — concurrent workers run a shared cooperation loop, self-assigning sub-tasks via a shared workspace instead of a controller dispatching them — and scaling from 1 to 1,024 agents lifted a pandoc-benchmark score from 33.89% to 55.06%, evidence the self-improving-harness pattern also scales horizontally, not just over time. (arXiv 2609.26781, Sep 2026.) Compilation: Growing Harness turns repeated agent-control logic into persistent executable code instead of having the LLM reconstruct it every task — cutting LLM calls 76.0–91.8% and inference cost 74.4–98.6% while letting smaller models hold performance, a concrete instance of this section's own "mechanized rules beat prose guidance" finding applied to the control loop rather than the compliance rule. (arXiv 2609.26760, Sep 2026.)

Distinguishing genuine multi-generation improvement from a lucky single child. Where the ablations above measure whether a harness self-improvement mechanism works, one metaharness fork adds a verifier for whether the claim itself is real: it requires lineage continuity across generations, digest integrity, frozen quality/cost thresholds, zero safety regressions, and two trusted reviewers per generation before crediting a multi-generation gain — explicitly not a claim that recursive self-improvement has been achieved, only that this specific improvement-claim shape can now be checked. Adversarially tested on 10,000 synthetic envelopes with zero false accept/deny. (ruvnet/metaharness #358, Sep 2026.)

Three practitioner instances from a YC panel, reported secondhand — cite the reporter, not an invented primary source. A widely-repeated framing from the panel: "the exact same model weights scored around 30% on ARC-AGI with a weak harness and roughly 95% with a better one" — an anecdotal restatement of this cluster's central claim rather than a new measurement. Three named systems: Prime Agent's /refine command revises "the agent's own supplemental prompts, memories, and subagent specs" without touching model weights, built around a persistent IPython REPL it uses as its core reasoning surface (a Recursive Language Model architecture). OpenJarvis names five architectural primitives for a personal-agent harness — local inference, persistent memory, on-device tool execution, privacy boundaries, cloud fallback — with a cloud model periodically tuning the local setup before handing control back to it; the panel's own claim, unverified beyond this report, is roughly 800x cheaper per routine task run locally versus the same task hitting a frontier cloud API. QM, the YC-internal fleet, runs "roughly 50 agents, built on the OpenClaw project, across accounting, legal, events, and engineering work" — described as "pulling the brain out of the sandbox," decoupling persistent reasoning/state from disposable execution environments, a fleet-infrastructure pattern adjacent to Fleet Engineering's crash-surviving identities. (explainx.ai, "YC Panel: Self-Improving Harnesses, OpenJarvis, and the Prime Agent /refine Loop", Sep 2026 — no primary source located for Prime Agent or OpenJarvis; treat the 800x figure as an anecdotal panel claim, not a measured benchmark.)

Correction from the primary source. QM's own README describes a multiplayer Slack/web agent harness where Pi, OpenCode, Codex, and Claude Code all drive one shared core — not "built on the OpenClaw project" as the secondhand panel recap above states. Every person and channel gets a scoped memory/files/keychain/sandbox, and an org picks one of three security postures (Strict/Auto/Dangerous) that narrower scopes can only tighten, never loosen. At nearly 15,000 stars and daily commits, this upgrades the citation above from secondhand-with-caveat to primary. (yc-software/qm, fetched Sep 2026.)

The cost case, quantified. A Hugging Face proposer/accept-reject loop that rewrote only the harness code around a frozen model matched Sonnet 4.6's legal-agent-benchmark score at roughly 7x lower inference cost — and with identical model and tasks, score ranged from 3.5% to 80.1% depending solely on harness quality. This is the sharpest evidence yet that harness spend, not model spend, is where the marginal dollar buys reliability — the same thesis as The Paradigm Shift, now with a cost multiplier attached. (Hugging Face, "Don't train the model, evolve the harness", Jul 2026.)

Literature map: the late-2026 arXiv wave, by question answered

Between late September and mid October 2026 about two dozen arXiv papers extended the cluster above. A paragraph each would bury the pattern, so the tables below group them by the question each one answers. The one-line contributions come from the papers' abstracts or listings, not from a paper-by-paper read, and every figure is the authors' own and unreplicated. For further candidates, Awesome-Agent-Harnesses is a small curated index (its title counts 208 papers and 67 production harnesses, essays and talks; updated 2026-10-14) organised by component (agent loop, tools, memory/context, multi-agent, evaluation, self-improvement loops) and framing the loop as one component of the harness. Treat it as a candidate list, not as evidence: its entries have not been reviewed here.

Harness² — the best-known entry, and the one that drops the verifier. Google Research's Harness² optimises the harness with model weights fixed. Per its README it exposes eight editable components (system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, plugins), runs a task-level Propose-Probe-Compose recursion, and keeps a persistent improvement harness that carries evidence-backed lessons from task to task. The README reports gains of up to 11.5 points on WorkBuddy-Bench and says a larger recursion budget helps further (self-reported). The paper frames it as two levels working without ground-truth verifiers: contrastive task-level edits, plus a higher level that generalises across tasks. An @akshay_pachaar summary (second-hand, not read at source) describes the same loop as proposing edits, running several candidate harness versions and composing a better one by comparing the runs, with no grader, executing via OpenCode or Codex. The contrast with Self-Harness and AHE above is the point: those gate edits on a held-out run or a verified prediction, whereas here the gate is a comparison between candidate runs. The sources do not establish whether that is as safe as a disjoint held-out gate (see Darwin Mode above), so treat it as the open question rather than a settled saving.

RunTune — control-measured harness changes from your own transcripts. RunTune reads Claude Code, Codex, Cursor and Antigravity transcripts, proposes capabilities (from successes), constraints (from failures) and routes, requires human approval, then measures each adopted change against a control and retires or revalidates it. Its docs/EVIDENCE.md reports a 150-day run over about 165k Claude Code attempts in which 51 of 154 candidates were proposed and 103 withheld, with a per-rule before/after net of the control (one guard rule at -19.1 points). Two of its own findings matter more than the headline: a model upgrade made 3 of 13 guard rules impossible to trigger (the removal test in practice), and sub-agent token columns had been inflated 38.9x by a logging bug that the check itself could not catch. All numbers are the author's, from one team's deployment, and the repo was created 2026-09-27 (9 stars at capture). It is a practitioner version of the gating discipline the papers below formalise.

Question Paper One-line contribution
When is an edit real? (admission and attribution) Self-Evolving Agent Harness Separates proposal from admission; gives a generalization bound, a regression-budget retention bound and a fresh-validation admission rule
Harness-Delta Attribution Splits a harness-evolution gain into artifact overfitting, test-time-scaling gain and residual generalizable gain; applied to 11 executors and three evolution frameworks
Compliance-bounded self-evolution (credit pipelines) Confines self-evolution to the runtime harness behind a hash-chained admission gate; in simulation (a simulated agent, not LLMs) it admitted 144 of 7,449 candidates, none worsening held-out error
VERSE Finds optimizer self-evolution fails without execution-based verification; the optimizer tests draft edits, replays failures and perturbs suspected steps before submitting
Trace-to-Architecture Diagnoses which component to change from traces, then validates the target gain, regression on successes and that the edit actually triggers; reports +6.2 to +30.3 points over the strongest baselines on ALFWorld, ScienceWorld and AppWorld
UCoE Evolves an executable verifier program together with harness state under fixed weights, testing verifier candidates on response contrasts with known preferences
How is the search run? Harmony Harness Multi-objective Pareto search with factorized operators separating general mechanisms from environment-specific adaptation
MESH-Harness LinUCB-scored recombination of module implementations under a limited evaluation budget
MILO Island-based multi-agent evolution co-evolving harnesses and the discovery strategy; rejected mutations count as negative evidence
Mixture of Self-Improving Branches Branches with evolving dev subsets and proposal policies to avoid single-trajectory local optima; builds on Meta-Harness
FreeEvolve Replaces the hand-built search loop with an editable, meta-evolved skill that decides what to test, how much evidence to collect and when to stop
ActiveSaddler Treats the training-scenario curriculum as a non-stationary bandit over reusable failure-pattern arms
Causal Improvement Graph Keeps improvement state in a persistent graph so prior findings govern later revisions instead of an ever-growing proposer history
ReHarness Backward revision of retrieved reflections from each new trajectory, under a strictly online streaming protocol
ScholarEvolve Draws on published research, not only observed failures, to guide a meta-coding-agent's harness evolution
What is evolved, composed or cut? Cortex Treats CLAUDE.md-style files, skills and rules as the harness; replays recurring developer corrections as tasks and keeps or deletes a rule only if repeated with/without runs show net gain (tested with Claude Haiku 4.5)
SHarP Saliency-based pruning of tools, instructions and support modules to cut the token overhead of accreted harness complexity with little performance loss
Agent Skill Evolution Empirical study of 2,608 first/last revision pairs across 3,159 SKILL.md files; most revisions change a rule or procedure; measures compliance effect on 21 models and four agents
ContextEvo Learns a context-management policy from long-horizon trajectories, starting from the open-source Pi-agent harness; reported comparable to or better than Codex, OpenCode and OpenClaw on three benchmarks
Harness Primitives Mines reusable primitives from failed trajectories and composes a task-specific harness at test time, because a global mechanism helps one task and hurts another
Meta-Skills A Builder learns meta-skills from a Target's execution feedback to construct harnesses for unseen tasks; +8.95 pp over no-skill construction on Harness-Bench/NewtonBench
Evo² Co-evolves model weights and harness with one policy as both Proposer and Reflector; see Learned Orchestration
What can go wrong? Provenance marks Published watermarks fail when a leaked harness passes through a self-evolver (KGW survives 0 of 12 copies); proposer habits decide survival; builds an attack-aware mark; see Agent Security Hardening for the wider threat model

Reading the rows together: the centre of gravity has moved from proposing better harnesses to admitting them, which is where the Darwin Mode, HarnessLens and EvoUndo entries above already sat. The attribution protocol is the sharpest new tool for a reader, because it asks how much of a claimed gain is the artifact overfitting to its own tasks. The pruning rows (SHarP, Cortex) are the self-improving counterpart of When to Remove Harness: the same measurement loop, run in the deleting direction.

Harness versus model: controlled comparisons. A second group of papers measures the harness against the model rather than optimising it, and the same conclusion keeps appearing.

Paper Result
Legislative document-to-XML Three harnesses by three Claude models on 153 bills: the weakest model in the full pipeline scored 0.948, the strongest model alone 0.806
MedicalHarness Controlled model-by-harness study on a 107-task benchmark that isolates individual harness mechanisms; argues an agent score belongs to the model-harness pair
Mingbird Attributes small-model agent failures to the harness and adds a prefill budget, a finish gate that re-reads the task before accepting completion, and signature-level loop detection
Box²-Bench Varies workflow reliability to test whether models can override misleading harness guidance; frontier models stay vulnerable, counterfactual SFT improves robustness, and outcome-based RL shifts models toward using helpful workflows
Corporate and investment banking Industrial case where LLM judges guided harness changes; tests whether the gain over direct generation is real or a grading artifact

Two cautions apply. The Box²-Bench result cuts against "more harness guidance is always better": frontier models stay vulnerable to misleading workflow guidance, so a harness is only as good as its guidance is reliable (see also Failure Patterns). And the banking paper's question (is the lift real, or does the judge reward the harness's style?) applies to every row above that scores itself with an LLM judge; see Verification. On the ARC-AGI-3 harness gap (above), an explainx.ai write-up reports Gemini 4 Argon at 97.5% with the provider-adapter harness against 49.8% with the standard one. That is a secondary source that says it could not load the leaderboard to confirm, so it is a pointer to check against ARC Prize, not a figure to cite.

Harness vs. Environment Engineering

Two complementary safety layers operating at different levels of the stack:

Layer Scope Controls
Harness engineering In-process settings.json permissions, lifecycle hooks (PreToolUse/PostToolUse), MCP gates, CLAUDE.md rules
Environment engineering Out-of-process OS user per agent, container isolation, network filtering, credential broker

The distinction matters for defense-in-depth:

  • Harness controls are enforced by the Claude Code runtime — effective against autonomous bad decisions, but inside the same process as the agent.
  • Environment controls are enforced by the OS or infrastructure — they limit what the agent can do regardless of what the model decides, and cannot be overridden by the model.

Three reference deployment patterns:

Pattern Harness posture Environment posture Use when
Approval-First All file writes in ask list Standard OS user New loop, unknown scope
Curated Allow-list Explicit allow list; everything else denied Standard OS user Loop scope is well-understood
Sandboxed Full-Auto Auto mode, full tool access Isolated container + network filter Fully autonomous production loop

Rule: start every new loop at Approval-First; promote to Curated Allow-list once approvals have become predictable/tedious for at least a week AND you can name the safe operations explicitly enough to write allow rules; promote to Sandboxed Full-Auto only once the OS-level isolation boundary is already in place and tested (not merely planned) and you've pre-agreed on how you'll detect and roll back a bad run.

See Permissions & Auto Mode for the full harness-layer control reference (allow/deny/ask lists, risk-tiered authorization, agent trust ramp).

(hidekazu-konishi, "Claude Code Harness and Environment Engineering", Apr 2026.)

Verification built into the harness (a separate verifier agent, objective evidence gates) is what makes a loop safe to scale up: you can run more iterations, more agents in parallel, and larger budgets only when each cycle's output is trustworthy.

The "Unstable Components" Design Axiom

"Models may speak like teammates, but they do not automatically gain teammate-grade stability, accountability, or sustained judgment."

This is the foundational posture for harness design: treat the model as an unreliable runtime component requiring containment, not a collaborator requiring instructions. The consequences for harness design:

  • Every agent output is unverified until a deterministic check confirms it
  • The harness enforces boundaries the model cannot override (hooks, OS-level isolation)
  • Stability comes from the harness, not from model capability — model improvement shifts the cost boundary but does not eliminate the need for containment

(wquguru/harness-books, AgentWay, Jun 2026 — the sentence is in book1-claude-code/locales/en/chapter-09-ten-principles.md § 9.1, not the README; a README-only check reports it missing.)

Generic Harness + Good Feedback Loop Beats Bespoke Scaffolding

A domain-specific evaluation of computer-aided-engineering (CAE) simulation agents found that a single-agent, generic multi-turn tool-use harness matches specialized systems built purpose-fit for the domain — and that execution-feedback repair (re-running the simulation and reacting to the actual error) beats scripted reflection (a fixed self-critique prompt) as the correction mechanism. This is direct evidence for When to Remove Harness's removal test applied one level up: not just "which components of my harness are load-bearing," but "do I need a bespoke harness for this domain at all, or does a generic one plus a real feedback loop already match it." (arXiv 2609.03718, "What Do CAE Simulation Agents Really Need Beyond a Generic Harness?", Sep 2026.)

Adversarial self-play as a harness-level robustness loop. A red-teaming system using self-play to generate multi-step adversarial attack paths and improve prompt-injection robustness applies the same verification-loop/adversarial-review pattern this doc documents for correctness (Self-Improving Harnesses) to model safety instead — the harness evolves against attacks the same way it evolves against task failures. (OpenAI, "GPT-Red: Unlocking Self-Improvement for Robustness", Jul 2026.)

Agent Tool-Choice Bias

How does an agent pick which library or SaaS tool to reach for when several would work? A controlled study — 5,292 validated sessions (of ~17,000 raw), 1,163 prompt variations, 75 repos, three agents (Claude Code, Codex, Cursor), simulated human-in-the-loop via Gemini 3.7 Flash — measured this directly rather than assuming brand recognition transfers. It doesn't, cleanly:

"Watching seventeen thousand tool choice sessions in the analysis undertaken, we saw twenty years of brand building carried out by tool vendors simply frozen in time. Agents reach for Docker the second containers come up, then draw a blank on the sandboxes it offers now... Your reputation follows you into the weights... but that weight operates under a different kind of gravity today."

— Theodore Otzenberger (Armature co-founder), quoted in The New Stack, "'Twenty years of brand building simply froze in time': How coding agents select their tools of choice", 2026-09-07

Concrete results: Resend won the TypeScript email-provider slot 55/89 sessions; SendGrid led Python (22/24), Postmark led Go (20/24), Azure Communication Services led Java (22/23). The three agents agreed on the same tool in only 42% of cells tested. Per-agent behavior differs sharply in how the choice gets made: Cursor bases its decision on a web search in ~2/3 of sessions; Codex searches the web almost always (94%); Claude Code relies primarily on training priors and searches only ~30% of the time, but browses roughly 3x more pages than Codex when it does. Claude Code also builds in-house nearly twice as often as Codex or Cursor (19% vs. 10%) rather than reaching for an external dependency at all. Being mentioned is not being chosen: PayPal was cited 139 times across sessions and selected zero times; LangChain was the most-mentioned framework (194 mentions) but chosen only four times.

Why this belongs in a harness doc, not a marketing one. Tool selection is itself a harness decision the agent makes autonomously, with real security and lock-in consequences (which SaaS gets your API keys, which library enters the dependency tree) — and it is driven by training-data prevalence and repo context, not the vendor's current pricing, feature set, or docs quality. A harness that constrains tool choice to an explicit allow-list (see Curated Allow-list above) removes this bias entirely rather than fighting it. See Agent Security Hardening for the supply-chain angle — vendors optimizing pricing/docs pages to bias agent tool choice is an emerging SEO-for-agents attack surface, not yet observed in the wild but structurally enabled by this finding.

Ledger Closure for Interrupted Tool Calls

When a session is interrupted mid-tool-call, external systems reading the session transcript get corrupted state unless ledger closure is enforced:

Every tool_use block must be paired with a tool_result block before the session can be considered consistently closed. An interrupted session that leaves a tool_use without a matching tool_result produces an uninterpretable trace — orchestrators that resume from this state make decisions based on corrupted input.

Applicability note: Ledger closure is primarily relevant for orchestrators that directly construct Claude API message arrays (custom harness, not CLI). Claude Code CLI sessions do not expose the raw message array; this pattern applies when you manage the conversation turn sequence programmatically.

Interrupt handling pattern: 1. Detect the interruption (timeout, crash, explicit stop) 2. Emit a tool_result for the pending tool_use with an error payload 3. Only then persist the session state to the state file

{
  "type": "tool_result",
  "tool_use_id": "<id-of-the-interrupted-call>",
  "content": "Interrupted — result unavailable",
  "is_error": true
}

(wquguru/harness-books, AgentWay, Jun 2026.)

Tool calls as bounded, cancellable state streams, not async functions. A harness-builder's field postmortem names the same tool-call lifecycle problem structurally: splitting a call into separate preview/execute/render-result phases forces every implementation to duplicate I/O and invent an ad hoc side channel for in-flight state — replaced by treating a call as one DOM-like element the executor mutates in place, with output-size limits built into the primitive itself rather than opt-in per tool. Validated against four architecture-test personas (multiplexed local workspace, remote driver, untrusted spectator, autonomous fleet) — the last forcing a host/sandbox boundary where only a single obedient stub (shell + grep) runs inside the untrusted VM and everything else (keys, session storage, policy) stays on the trusted host. (Stencil, "The Harness Playbook", Sep 2026.)

A related resumption failure: at-least-once delivery duplicating committed work. A stalled forward-loop iteration that gets retried can re-deliver the same external item (e.g. a billing transcript) a second time — omnigent traced roughly 20% of a billing duplication bug to exactly this, and fixed it not by making retries impossible but by making the write idempotent: external items are persisted under a UUID5 key derived from the source ID, so a duplicate delivery becomes a no-op rather than a second commit. The same principle as ledger closure above — a resumed session must not fabricate progress that didn't happen — applied to the loop's output rather than its transcript. (omnigent-ai/omnigent #6654, Sep 2026.)

The Two-Part Harness (Anthropic Engineering)

Anthropic's "Effective Harnesses for Long-Running Agents" defines a two-role harness split:

1. Initializer Agent

  • Reads the high-level goal and generates a structured JSON feature list — a machine-readable work plan
  • Sets up session state: which files are in scope, what the success criteria are, what constraints apply
  • Writes a mandatory session init routine that the coding agent reads at the start of every context window

2. Coding Agent

  • Executes tasks from the JSON feature list, one unit at a time
  • Uses browser automation testing to verify UI/UX changes without requiring a human
  • Applies git-based recovery: commits after each unit so any crash can resume from the last known-good state

The key invariant: the initializer runs once; the coding agent runs many times, each time within its own context window, always starting by reading the session init file.

The Four-Type Loop Taxonomy (Claire Vo / Lenny's Newsletter)

Every agent loop has a trigger type. Choosing the wrong trigger type is one of the most common harness design mistakes:

Type Trigger Notes
Heartbeat Fixed interval (every N minutes) Checks for work; exits immediately if nothing to do. Cheap if work is rare; expensive if the interval is too short
Cron Scheduled time (daily, weekly) Predictable resource usage; suits batch jobs and digests
Hook Event-driven (push, PR, webhook, file change) Only fires when work exists — no polling overhead. Preferred for reactive workflows
Goal Runs until a success condition is met Hardest to write; most likely to burn tokens without output. Requires a rigorous stop condition and budget cap

The goal loop is the most powerful and the most dangerous: without a verifiable stopping condition and a hard spend cap, it will run indefinitely.

See also: Loop Contract for mandatory BUDGET and STOP properties.

The Official Claude Team Loop Types

Anthropic's own Claude Code team publishes a complementary four-way split, framed by how much human involvement the trigger requires rather than by mechanism:

Type Trigger Stops when Best for
Turn-based A user prompt Claude judges the task complete or needs more context Short, exploratory work
Goal-based (/goal) A manual real-time prompt Goal achieved OR max turns reached Tasks with verifiable completion criteria
Time-based (/loop, /schedule) A fixed interval You cancel it, or the work completes Recurring tasks, monitoring external systems (/loop runs locally, /schedule moves it to the cloud)
Proactive An event or schedule, no human in real time Each task exits on completion; the routine runs until manually stopped Well-defined recurring work (bug triage, dependency upgrades)

This taxonomy overlaps but doesn't map 1:1 onto Heartbeat/Cron/Hook/Goal above — it splits out turn-based (conversational, human-prompted) as its own category, and folds Heartbeat+Cron together into time-based. Use Claire Vo's table to pick a trigger mechanism; use this one to decide how much of the loop should run without a human in the room. ("Getting started with loops", claude.com/blog, Jun 2026.)

Three-Agent Full-Stack Harness (Anthropic Engineering)

For complex, multi-feature applications, Anthropic extended the two-part harness into a three-agent system (Prithvi Rajasekaran, "Harness design for long-running application development", Mar 2026):

Agent Role Key behaviour
Planner Converts 1–4 sentence prompts into detailed product specs Ambitious on scope; avoids technical over-specification; identifies AI feature opportunities
Generator Implements features from spec Self-evaluates before QA handoff; uses git for recovery; works in sprint contracts
QA / Evaluator Active testing with Playwright MCP Tests UI, API endpoints, and database states like a real user; grades against a negotiated set of testable criteria, each with a hard threshold

The Planner prevents cascade errors from spec mistakes by staying high-level. The Generator negotiates sprint contracts with the Evaluator before each build phase.

Sprint Contract

Before each implementation sprint, the Generator and Evaluator negotiate a specific set of deliverables and testable criteria, granular enough that Sprint 3 of Anthropic's own run carried 27 of them for the level editor alone:

Sprint N contract:
- What will be built: [specific features]
- Success criteria: [testable, objective conditions; each with a hard threshold]
- "Done" definition: all criteria pass in QA

This bridges the gap between high-level user stories and implementation detail without over-constraining technical decisions upfront. It also eliminates ambiguity about what the Evaluator is checking — the criteria are agreed before code is written.

Load-Bearing vs. Optional Components

As models improve, harness components that were essential scaffolding become unnecessary overhead. Re-evaluate which components are load-bearing with every significant model release:

  • With Opus 4.5: sprints, explicit decomposition, and per-sprint evaluation were essential
  • With Opus 4.6: model capability increased enough that sprint removal did not degrade output; single end-evaluation often sufficient

Rule: find the simplest solution possible, and only increase complexity when needed. An evaluator only adds value when the task sits beyond what the baseline model handles reliably solo. As that boundary moves outward with each model generation, periodically simplify your harness and measure whether quality holds.

Starving the Reviewer of Familiarity on Purpose

A concrete instance of context isolation applied specifically to a code reviewer: a production 71-skill/34-hook/31-agent monorepo pins its edge-case-hunter agent to a fixed model/effort tier (Sonnet, effort:high) and sets omitClaudeMd: true on its adversarial-reviewer agent so the project's own CLAUDE.md conventions are not auto-injected into the reviewer's context. The stated reasoning: a reviewer that already knows "why we do it this way" tends to rationalize familiar patterns instead of questioning them — deliberately reducing context here is what forces it to actually find issues. This KB has only one deep-read of this repo's content on file so far (this finding); it is otherwise still a tracked-but-unread source (see KB_GAPS.md). (milosCvetkovicDev/claude-code-monorepo, Sep 2026.)

A Maker/Checker Harness Beyond Software: Lab Equipment

The multi-role split above generalizes past coding. Anthropic's own Model Hardware Standard research preview runs the agent loop as four distinct roles, each a fresh Claude instance, driving physical lab/manufacturing equipment rather than a codebase — a maker/checker structure with closed-loop error recovery and no human intervention required mid-run. It is evidence that this doc's harness patterns (role separation, verification before commit, fresh-instance-per-role) are not coding-specific design choices; they recur wherever an agent loop drives a system with real-world side effects and a real cost of being wrong. (Anthropic, "Model Hardware Standard (research preview)", Sep 2026.)

Dynamic Workflow Patterns (Anthropic Engineering)

Rather than fixing one harness shape per project, Claude Code can write its own custom multi-agent harness per task by selecting from six named orchestration patterns — the harness itself becomes a decision the agent makes, not just a human-authored default:

Pattern Shape Guards against
Classify-and-act A router classifies the task, then dispatches to a matching specialist Applying the wrong workflow shape to a task that didn't need it
Fan-out-and-synthesize Independent workers cover sub-parts in parallel; a synthesizer merges results Sequential work that could have been parallelized
Adversarial verification A separate agent tries to break the work rather than confirm it Self-preferential bias — an agent approving its own output
Generate-and-filter Many candidate solutions generated, weak ones filtered before any single one is polished Sunk-cost commitment to the first plausible draft
Tournament Candidates compete head-to-head across rounds; a judge advances the strongest Local optima from evaluating candidates independently rather than comparatively
Loop-until-done Repeat generate → evaluate until a stopping condition is met Goal drift — treat this as the canonical implementation of the Loop Contract's STOP property

These are explicitly framed as countermeasures to three named failure modes: agentic laziness (settling for a shallow but plausible-looking pass), self-preferential bias (an agent trusting its own output more than external evidence), and goal drift (the task definition eroding across iterations — see Failure Patterns → Context drift). Pick the pattern to match the failure mode you are most exposed to, not by default habit.

(Anthropic, "A harness for every task: dynamic workflows in Claude Code", Jul 2026.)

Cross-Model Division of Labor

A recurring pattern across mid-2026 harness design: split roles across models by cost/quality tier, not just by function. A stronger (and more expensive) model is kept for specification, review, and destructive operations; a cheaper or faster model does the bulk of implementation — inverting the naive default of running one model for everything.

  • Advisor loop: an executor session calls a stronger model only on demand for guidance, while a cheaper model does the bulk implementation work — the stronger model is consulted, not driving. (@steipete, Jul 2026.)
  • Decision-model routing inside the harness: OpenClaw's next version "uses a decision model to automatically decide between steer or queue", that is, whether a new user message interrupts the running agent or waits behind it. It is a lab feature, and the supported variants are listed as "Jef* and API-compat (e.g. local models like Kef...) and ONNX variants" (@steipete, Sep 2026). The user keeps a manual override for a wrong call: "You can always press the steer button on a queued message" (@steipete). Earlier, asked whether he would "intagrate natively jev to openclaw", he replied "We already did! For queue/steer, thinking level, skill search etc" (@steipete, Sep 21 2026). Asked "Any plans to leverage decision models for compaction and tool calls as well?", he replied "defo for tool search and skills!" (@steipete) — a plan, not a shipped feature, and compaction is not addressed. The pattern: a cheap model makes the harness's routing decisions, and a human override stays one click away. The decision model is plausibly the Jev that MCP Security benchmarks — Steinberger's own Sep 21 answer says Jev is integrated for queue/steer — but whether it is the model behind the Sep 23 steer-or-queue feature is not stated.
  • codex-first SKILL.md: a Claude Code skill formalizing this as a hard rule — Claude keeps design, review, and destructive operations; implementation is delegated to codex exec --yolo via temp-file specs, with escalation back to Claude after two failed delegation attempts (a bounded retry, not an infinite handoff loop). (steipete/agent-scripts, Jul 2026.)
  • Puppetmaster: generalizes the pattern into a provider-neutral control plane — a supervisor in front of multiple agent CLIs (Cursor, Claude Code, OpenAI) with leased worker subprocesses and typed, deterministically-stitched artifacts, so the division of labor is enforced by the control plane rather than by convention. (professorpalmer/Puppetmaster, Jul 2026.)
  • Unified Harness Protocol (UHP): an open standardization attempt in the same direction — a self-hosted router exposing five harnesses (Codex, Claude Code, Hermes, Pi, DeepSeek Harness) behind one OpenAI-compatible API, with concurrent isolated-workspace sessions, SSE streaming, and cancellation. Notable for shipping a runnable, versioned conformance suite (uhp-conformance --class full) against a public spec, not just a wrapper library — 664 stars. (HarnessRouter/harnessrouter, Sep 2026.) The figures above are the September snapshot; the project has since grown to 20+ harness bases — see A protocol at the harness boundary.
  • Three independent large meta-harness/harness-router products converged in the same week (1,286–10,316★ each), a dominant-theme signal worth naming as its own emerging category rather than three unrelated finds. openrig frames the layer explicitly — "a harness wraps a model. A rig wraps your harnesses." — with YAML-defined multi-harness teams (Claude Code + Codex in one "rig"), tmux-backed persistent "seats," and an owner/checker quickstart pairing two Codex seats, one implementing, one verifying. orchestra (15★, smaller but conceptually clean) independently arrived at this KB's own worktree-isolation + cross-model-review combination: a goal-driven orchestrator spawns workers each in an isolated git worktree, routes worker output to a different model for review, and merges only what passes — workers persist for hours or days rather than one request. Alongside omnigent (above) and HarnessRouter's UHP, this makes four unrelated teams shipping the same "one control layer over many coding-agent CLIs" shape in one sweep — evidence the pattern has moved from novelty to category, not proof any one implementation is the reference. (mvschwarz/openrig; DrSeedon/orchestra, Sep 2026.)
  • Official first-party version: OpenAI's own Claude Code plugin implements the same reviewer-executor separation as a supported product, not a community skill — a /codex:adversarial-review command and an optional gate that blocks Claude's response pending Codex's validation. Notable because the pattern now has official backing from a different vendor, not just community consensus. (openai/codex-plugin-cc, Jul 2026.)

  • claude-uds-bridge — bidirectional peer messaging, not sequential handoff. Reverses codex-plugin-cc's reviewer-gate direction: a running Codex task registers as a live peer inside Claude Code's own session registry over a documented Unix-domain-socket protocol, answering Claude's queries mid-turn instead of being invoked as a sequential post-hoc reviewer. (LeonKohli/claude-uds-bridge, Sep 2026.)

  • HydraFusion: GitHub Copilot's own multi-model orchestration research preview, routing across Single/Cascade/Critique workflow shapes rather than picking one model per session — GitHub's announcement claims 4.9pp higher quality than Claude Opus 5 at 67% lower workflow cost on TerminalBench 2.1. Treat this as a competing vendor's self-reported benchmark rather than independently verified — but it is a first-party instance of a competitor betting on orchestration-shape choice, not model choice, as the lever, which corroborates this section's thesis regardless of whose numbers are right. (GitHub, "Project HydraFusion"; explainx.ai coverage, Sep 2026.)

  • Mechanically-enforced review-before-merge: rather than trusting the orchestrating agent to run cross-model review before merging a worker's isolated-worktree branch, the merge operation itself blocks unless a review receipt matches the exact snapshot's session ID — the enforcement lives in the merge code path, not in a convention the agent could skip under time pressure. (DrSeedon/orchestra, Sep 2026.)

  • Vibe Dev v9 — a physically write-denied verifier. Per-stage model tiering (a higher tier owns planning, critique, and verification; a cheaper tier does bulk implementation), taken one step further than the advisor/codex-first pattern above: the verifier role is denied Write/Edit tool access at the harness level, so it cannot fix what it finds even if instructed to — only report it. Its stated completion bar, "done = proven by a real run," requires screenshots, execution traces, or mutation-testing evidence, not a passing typecheck. (andrewcigan/vibe-dev-plugin, Sep 2026.)

  • Hearting — one contract projected across three native runtimes. Rather than a provider-neutral router (UHP) or a control-plane supervisor (Puppetmaster), Hearting turns a natural-language request into a structured "route card," assigns it a sealed per-node model tier (deep / balanced-deep / light / mini), and projects a single harness-manifest.json onto whichever of Claude Code, Codex, or OpenCode's native surfaces is running it — the division of labor is declared once and reinterpreted per runtime, instead of routed through a shared API layer. (dmlguq456/hearting, Sep 2026.)

A later Hearting release moves the wait-vs-end-turn decision out of prose and into the receipt itself: a parent_next=end-turn|bounded-wait field tells the calling agent what to do next, with unrecognized delivery states failing closed to a bounded wait rather than silently ending the turn — closing a class of bug (missed arming, needless polling) that prose instructions had produced before. (dmlguq456/hearting, commit c0b311d0, Sep 2026.)

A further release adds checkpoint manifests published mid-cycle: an open work cycle writes an interim manifest whose artifact IDs the final seal reuses rather than reissuing, so a cycle interrupted mid-flight leaves IDs a resumed run can match against rather than orphaning them. (dmlguq456/hearting, Sep 2026.)

A further fix recovers a specific completion-tracking edge case: when a recorded artifact is later moved, Hearting cryptographically re-verifies it at its new location and converts what used to read as an "artifact-missing" failure into a PASS verdict once the completion event, placement record, and file content all reconcile — a stopping- condition fix distinguishing "the work is done but the file moved" from "the work never finished." (dmlguq456/hearting, Sep 2026.)

This is the same underlying idea as Subagents' "strong eyes, cheap hands" cost-asymmetric role allocation, generalized from same-vendor subagents to cross-vendor sessions — the review/execution split survives the model boundary. It is also the cost-allocation side of the same coin as Verification's cross-model independence pattern 5: that section argues cross-model review is more effective (different blind spots); this section is about why it is increasingly also cheaper (cheap model implements, expensive model only reviews/advises on demand).

Five-Wave Execution Model

A typed sequential execution pattern where agents deploy in parallel within each wave and output feeds the next gate:

Wave Role Mode
1. Discovery Read-only audit — gather context, identify scope Read-only
2. Impl-Core Primary implementation, parallel agents Write
3. Impl-Polish Edge cases, integration, secondary paths Write
4. Quality Simplification pass — before any test authoring Write
5. Finalization Commit, create carryover issues for incomplete items Write + commit

Wave 4 (Quality) is an inversion of standard TDD: a dedicated simplification pass runs on AI-generated code before tests are written. This prevents tests from cementing suboptimal implementations — once tests pass against an awkward structure, that structure becomes load-bearing. Simplify first; then write tests against the simplified code.

Between waves: a confidence-scored reviewer audits deliverables across multiple dimensions. Findings at ≥80% confidence surface in the main report; findings at 50-79% move to a separate "Possible Issues" section for human review rather than being dropped; only findings below 50% go unreported. (See Subagents for confidence-scored gates.)

(session-orchestrator — Kanevry/session-orchestrator, Jun 2026.)

Runtime Republic vs. Constitutional Control Plane

Two fundamentally different harness philosophies, each suited to different contexts:

Runtime Republic (Claude Code) Constitutional Control Plane (Codex)
Authority source Emerges from the dominant query loop — continuous negotiation with reality Encoded upfront in types, policies, event systems
Decisions Flow from conversation and context Flow from the constitution
Flexibility High — instructions and context steer behaviour Low — constitution is fixed at deploy time
Predictability Medium — model reasoning introduces variance High — policy violations are structurally impossible
Best for Exploratory, creative, or complex tasks Regulated, auditable, compliance-sensitive workflows

Neither is strictly better — the choice depends on how much variance you can accept and how much authority you need to encode upfront before the loop runs.

(wquguru/harness-books, AgentWay, Jun 2026.)

Control-Plane / Execution-Plane Split (kernel-gated mutation)

A stronger version of the constitutional control plane: rather than encoding policy in types the agents should obey, route all state mutation through a kernel that is the only authorized writer. Skill agents are read-only over raw state and may write exclusively through kernel subcommands; the kernel owns the state machines, budget enforcement, circuit breakers, integrity logging, and session handoff. A multi-phase loop (discovery → triage → make/review/integrate → handoff) then persists across Claude sessions via content-hash-anchored state and append-only event logs, with human-approval gates on irreversible actions.

The difference from a plain allow-list: the agent physically cannot corrupt state because it has no write path except the kernel's audited subcommands — governance is enforced by the architecture, not by the model's compliance. This is the harness-level counterpart to the repo-owned durable ledger in Memory Patterns. (Sungmin-Cho/claude-deep-loop, Jul 2026.)

Feeding the kernel's own history back into itself. A follow-up release added an insights kernel subcommand that mines the kernel's own chain-verified run history into deterministic insights — feeding the proposal step of the next loop iteration and the init step of the next loop entirely. This closes a hill-climbing feedback loop inside the control plane: the harness doesn't just execute a fixed proposal step, it proposes its own next move based on evidence from its own audited past runs, without loosening the kernel's read-only-for-skills invariant (insights are emitted via the same atomic-write + append-anchored-event path as any other kernel write). (Sungmin-Cho/claude-deep-loop v1.4.0, Jul 2026.)

A further release hardens pause/resume: a needs-human pause now resumes with the same owner and workstream affinity preserved under budget/breaker/lease fencing, and an abandoned inline-maker attempt is treated as cancelled work rather than silently blocking the whole goal from completing. (Sungmin-Cho/claude-deep-loop, v1.24.1, Sep 2026.)

Brain, hands and session as separate interfaces (Anthropic, Apr 2026)

Anthropic's Managed Agents post splits an agent into a session (an append-only event log kept outside the harness), a harness (the loop that calls Claude and routes its tool calls) and a sandbox. The harness calls a sandbox like any tool, execute(name, input) -> string, so a dead container is a tool-call error, and a crashed harness restarts with wake(sessionId) from the last logged event: state survives a crash because it was moved out of the harness. Credentials moved too: tokens are kept "never reachable from the sandbox" (see Agent Security Hardening). Anthropic reports p50 time-to-first-token down roughly 60% and p95 over 90% from provisioning containers only when needed (its own measurement, no method given). (Anthropic Engineering, "Scaling Managed Agents: Decoupling the brain from the hands", 8 Apr 2026; read in full 2026-10-19.)

The Query Loop as System Heartbeat

A book-length treatment of Claude Code's own runtime argues the query loop — not the model, not any single tool call — is the unit that determines system maturity. Analysis of the published npm package identifies nine tracked state variables assembled into one State object per conversation (messages, toolUseContext, autoCompactTracking, maxOutputTokensRecoveryCount, hasAttemptedReactiveCompact, pendingToolUseSummary, stopHookActive, turnCount, transition), and a strict pre-model input governance pipeline that runs before the model ever sees a token: memory prefetch → skill-discovery prefetch → truncate to compact-boundary → apply tool-result budget → history snip → microcompact → context collapse → autocompact attempt. The thesis: governing input is the runtime's job, done before inference, not left for the model to sort out mid-turn.

Layered recovery follows the same ascending-cost-and-destructiveness principle this KB already applies elsewhere: a prompt-too-long error tries context collapse first and only escalates to a full reactive compact if that's insufficient; a max_output_tokens cutoff first raises the token cap, and only once the cap is already maxed does it append a continuation message telling the model to resume from where it was cut off — never an apology or a re-summary.

The analysis catalogues eight distinct termination/recovery modes in the stop-condition matrix: stream ends with a pending tool call (follow-up), stream ends clean (stop hooks fire), user interrupt (abort + synthetic tool-result for orphaned calls), recoverable prompt-too-long (collapse, escalating to compact), output-cap-not-yet-maxed (raise and retry), output-cap-already-maxed (continuation message), a stop-hook block recurring after compact was already attempted (an explicit "double failure" guard that skips the hook and surfaces the error directly rather than looping forever), and an API error (return immediately, no retry). Compare this granularity to the Stop Condition Taxonomy — the same idea, at the level of one runtime's actual state machine rather than a design abstraction.

Caveat: this is a third-party technical analysis of Claude Code's published package (chapter cites specific file/line references, e.g. query.ts:203-217), not an Anthropic disclosure — treat the state-variable names and line numbers as this analysis's own findings about a specific build, not a stable public API. (Harness Books — AgentWay, undated, fetched Sep 2026.)

Ten Principles of Harness Engineering

The same book distills its analysis into ten principles, several of which this KB already documents under different names — listed here as a compact index, cross-referenced rather than restated:

  1. Treat the model as an unstable component, not a colleague — see The "Unstable Components" Design Axiom
  2. The prompt is part of the control plane, alongside runtime, tool schema, memory, and hooks
  3. The query loop is the heartbeat of the agent system — see above
  4. Tools are managed/governed execution interfaces, not raw function calls
  5. Context is working memory — governability matters more than quantity — see Context Management
  6. Error paths are the mainstream paths — prompt-too-long, max-output-tokens, interrupts, hook loops, and compact failures are daily weather, not edge cases, and must be designed for up front
  7. The goal of recovery is to keep working, not to apologize
  8. Multi-agent's purpose is partitioning uncertainty — research/implementation/verification/synthesis run in separate containers, recombined by a coordinator — see Subagents
  9. Verification must be independent — the system cannot grade its own homework — see Verification
  10. Team/organizational process matters more than individual skill — layered CLAUDE.md, explicit approvals, executable skills, lifecycle hooks, traceable transcripts, and a unified definition of "verified" across the team

(Harness Books — AgentWay, "Ten Principles of Harness Engineering", undated, fetched Sep 2026.)

A distinct, primary-adjacent source: the same author's twelve-thesis anthology. Ryan Lopopolo — the OpenAI engineer whose harness-engineering case study is cited near the top of this doc — maintains a separate twelve-thesis anthology (2,682 stars) organizing harness engineering around worker-constancy, just-in-time context routing, tool legibility, authority-vs-capability separation, and turning recurring corrections into durable infrastructure; distinct from the Ten Principles above (different author, a practitioner's essay collection rather than a third-party code analysis) and explicitly the author's own extension of, not independent support for, the OpenAI post it cites as its primary source. (lopopolo/harness-engineering, fetched Sep 2026.)

Object-Oriented / Code-as-Action Agents (NOOA)

A third alternative to the persistent-orchestration-graph and event-driven/serverless patterns above: model agents as Python objects rather than prompts or graph nodes. NVIDIA-labs' NOOA (NVIDIA-labs Object-Oriented Agent framework) gives a class's docstrings-as-prompts and lets methods with a literal ... body be completed at runtime by an LLM-driven agent loop, while methods with a normal body stay ordinary deterministic Python — so control flow, state, and non-agentic logic are just code, and only the genuinely agentic pieces are model-driven. The framework names six model-facing capabilities it unifies onto one object: typed input/output, pass-by-reference over live objects, code-as-action (the model acts by writing Python in a Jupyter-style REPL with access to self, imports, and helpers), "programmable loop engineering," explicit object state, and model-callable harness APIs — replacing prompt templates, tool schemas, callbacks, and workflow graphs with one substrate.

Reported results: 82.2% on SWE-bench Verified with GPT-5.5, at roughly half the token cost of competing harnesses; the paper additionally reports on Terminal-Bench 2.0 and ARC-AGI-3. Fully open source, no GPU/NVIDIA hardware required — any local or API-based LLM works. The open-source implementation ships as separate packages: nooa-cli (CLI + trace viewer), nooa-acp (an Agent Client Protocol implementation for hosts like Zed), nooa-memory, and nooa-bench (a Harbor benchmark runner) — Apache 2.0, 1,991 stars, daily commit cadence as of Sep 2026. (Cobus Greyling, Aug 2026; arXiv 2607.20709, Jul 2026; NVIDIA-NeMo/labs-OO-Agents, fetched Sep 2026.)

Harness-Agnostic Projection

A harness-agnostic design separates loop logic from the CLI or platform it runs on. The pattern: define all agent roles, tools, and workflows in a single source directory (.apm/ or equivalent), then compile that source to the layout required by each target harness (Claude Code, Codex, Copilot, Cursor, Gemini, Kiro).

Benefits: - Portfolio-level consistency: same security, verification, and escalation policies apply across all harnesses - Avoid lock-in: swap or add harness targets without rewriting loop logic - Specialist agents (adversarial reviewer, security analyst) ship as portable units usable in any target harness

A lighter-weight version of the same idea: instead of compiling to each target's native layout, put the control flow itself (decompose → fan-out workers → aggregate → review gate, looping on a negative review) in ordinary code behind one adapter interface (Agent.run()), so any backend — Claude, Codex, opencode, aider — plugs in without the harness knowing which one it's talking to. The same orchestration then exposes itself two ways: as an MCP server (callable from Claude Code, Cursor, Cline) and as a standalone CLI, with long-running MCP calls returning a run_id immediately and polling a cursor-based tail rather than blocking a request past its timeout. (luckeyfaraday/athena-loops, Jul 2026.)

The cost of neutrality: a counter-argument, not a measurement. Harrison Chase's case, as quoted by Harry Stebbings (17 Oct 2026), is that a harness "can be very tied to the model, or it can be neutral" and that tying memory to the harness and the harness to the model leaves you "kind of stuck". swyx replies that "kv cache and dreaming optimizations" are what "only harness-model colocation can achieve"; no data is given. The measured piece nearby: each model keeps its own prompt cache, so a mid-session switch recomputes the request (Cost Control).

Security review at specification stage: In a harness-agnostic design, a dedicated security agent reviews the compiled harness specification before any implementation begins — not after. Fixing a security gap at specification costs 1×; fixing it post-implementation costs 10×+. (eugenelim/agent-ready-repo, Jun 2026.)

The .apm/ primitive manifest — Monad-Harness does not define its own primitive schema; it adopts APM (Agent Package Manager), Microsoft's runtime-agnostic authoring format, as its "compilation substrate." APM's .apm/ format defines seven primitive types, not six:

Primitive Contents
Skills Self-contained capability bundles (SKILL.md + scripts + assets)
Prompts Reusable prompt templates with frontmatter
Instructions Long-lived behavior rules (style guides, conventions)
Agents Personas with explicit scope, tools, and triggers
Hooks Event handlers fired by the runtime (pre-commit, on-tool-use, …)
Commands Slash-command shortcuts the developer types into the agent UI
MCP servers Tool-server declarations consumers can wire into their harness

apm compile reads .apm/ and writes deterministic per-target output for each supported runtime (.claude/ for Claude Code, .codex/ for Codex, etc.). Primitive files contain no CLI-specific directives — portability is enforced by convention, not tooling.

(sergiocarvalhosa/Monad-Harness, Jun 2026 — a proposal/spec-stage repo (PROPOSAL_L0_v2.0.md, SPEC_V1.md, ROADMAP_v1.md; no shipped code as of this writing) that builds a lifecycle-management layer on top of APM rather than authoring its own schema.)

A protocol at the harness boundary: UHP and HarnessRouter (Sep–Oct 2026)

Projection compiles one source to many harnesses. The Unified Harness Protocol (UHP) takes the other route: leave each harness whole and standardise the boundary in front of it. A Daily Dose of Data Science write-up separates harness routing (choose the agent runtime that owns the whole task) from model routing, and describes UHP (version 2026-09-12 at that time) as a Responses-API-shaped HTTP contract with SSE events, sessions, files and cancellation, with the self-hosted HarnessRouter running one workflow through Codex and Claude Code. The project's ecosystem page places UHP as the product-to-complete-agent-harness boundary (tasks, sessions, streaming, files, results), with the harness keeping its own agent loop and tools, and MCP, A2A, ACP and AG-UI as neighbouring layers. A related academic proposal sits one layer down: HEAR is a bidirectional harness-engine protocol in which the harness communicates workflow intent and the inference engine returns runtime state and capabilities.

The repo's October commits are useful less as a feature list than as measured examples of supervisor-layer design. All are the project's own records:

  • A step budget enforced outside the harness. Twelve harness bases (including codex, pi, qwen, gemini, cline, opencode and kilo) had no step limit of their own, so a bounded task ran to the wall-clock cap. The fix counts tool calls by id, stops the process as if cancelled once past the budget, and ends the turn error_max_turns, which reports the task as incomplete and says where to raise the limit. A BUDGET/STOP control from the Loop Contract implemented by the router, not by trusting each harness.
  • A reasoning-effort knob in the contract. v0.30.0 adds a per-task reasoning.effort (none to xhigh) that works across 19 of 20 harness bases, clamps to the nearest level the model really has, and reports requested versus applied effort and reasoning_tokens in usage.
  • Stall limits tuned from measurement, and one tuning reverted. v0.31.0 cut the silence timeout on a provider call from 10 to 3 minutes, having measured some models sending nothing for 201-211 s at the highest thinking level, but v0.31.1 took it back about two hours later (10 minutes again) because a wider measurement showed the silence was not limited to those routes.
  • Verify before offering. v0.29.0 and v0.33.0 add nine more harness bases (including Qoder Cloud Agent as the first remote runtime), publish per-harness pass rates on five console scenarios (e.g. Kilo 291 of 295 runs), and withhold any model whose own failure repeated. jcode's own todo and memory tools were switched off because they led to runaway "re-validating" turns, a concrete runaway-loop failure mode.
  • Memory and environment as protocol chapters. The spec's changelog adds an Environments chapter (2026-09-28: build once, read-only mount) and a Memories chapter (2026-10-04: ordered text/file parts, role-tagged observed turns, a memory_unsupported refusal, conformance checks ME-01 to ME-10; implemented in v0.34.0 on 2026-10-15). The 2026-10-19 revision lets a new task import earlier turns as session history, stamps turns with created_at and completed_at, and lets a server retire a harness's sessions under a stated restore window (the one non-additive change). It also records that conformance suite post1 caught the reference server answering running where the lifecycle table requires in_progress: a suite check that had been too weak to see a status mismatch.
  • A default that means failure. A reconcile sweep read a missing heartbeat as 0 and aged a newborn session from 1970, settling it failed at once. The fix floors age by created_at and leaves a session of unknown age alone ("an unknown age is past no cap"). It is the same defect class this repo's own pipeline has paid for: an absent value must not default to a verdict.

A vendor post (no permalink) claims Qoder from Alibaba Cloud has adopted UHP; the only corroboration in the sources read is the v0.33.0 release above, which lists Qoder Cloud Agent as a supported remote runtime. Neither establishes that Alibaba itself adopted the protocol.

8-Phase DAG Execution Model (Tenet)

An extension of the five-wave model for harnesses covering 12+ hour development cycles:

Phase Role
0. Bootstrap Load goal, context, and existing state
1. Interview Clarify ambiguities; gather constraints before any code is written
2. Spec & Harness Produce a typed, reviewable specification (not code)
3. Visuals Design/mockup pass if UI is in scope
4. Decomposition Break spec into DAG of parallelisable tasks
5. Execution Implement tasks; each task assigned to one agent context
6. Evaluation Independent critic pass per deliverable
7. Agile Retrospective; carry incomplete items forward as first-class work units

3-Critic Pipeline: The evaluation phase deploys three independent critics, each running in a fresh context window with no access to the original implementer's reasoning — only the output artifact. Fresh context prevents critics from reasoning from the same anchors as the implementer. Each critic scores independently; disagreements surface boundary conditions.

Steer Message Taxonomy: Mid-run course corrections use a typed taxonomy rather than freeform messages, to prevent loop breakage:

Type When to use Effect
context New information the agent needs (API changed, requirement clarified) Adds context; does not redirect
directive Explicit redirect to a different approach Reprioritizes the job queue; the in-flight job continues rather than being cancelled
emergency Safety or security concern requiring immediate halt Stops current execution; escalates

In tenet's own implementation, injecting a directive steer reclassifies and reprioritizes pending work but does not itself cancel or roll back an in-progress job — treat any specific data-corruption claim about mid-subtask steering as unverified against the source.

(JeiKeiLim/tenet, Jun 2026.)

Meta-Harness: 3-Tier Policy Hierarchy

A meta-harness governs multiple sub-harnesses (Claude Code, Codex, Cursor) under a unified policy layer. Policies are layered in three tiers; per Omnigent's own model the end-user session tier evaluates first, and any DENY there short-circuits before the agent-spec or server-wide tiers run — the reverse of a simple "last tier wins" cascade:

Tier Scope Typical controls
Server Organisation-wide Spend limits, denied tool categories, audit logging policy
Agent Per-agent-type Allowed tools, permission mode, model selection
Session Per-invocation Task-specific overrides, context injection, budget adjustment

Harness-swap without state loss: Switch the underlying CLI (Claude Code → Codex, or vice versa) mid-project by externalising all state to standard files (GOAL.md, STATE.md, CLAUDE.md) that any harness can read. The agent's context resets; the project state persists.

Cross-device session continuity: Serialize the session ID and connection parameters at the start of each run. Any device or runner that has the session ID can resume the session without re-establishing context from scratch.

Compaction persistence — context compaction events are persisted alongside the session state. When a session resumes (claude --resume), the harness replays the compaction log to reconstruct the effective context without requiring the agent to re-read all prior files — reducing resume latency significantly on long sessions.

Spec reconstruction on resolve-miss — Omnigent's sub-agent spec resolver reconstructs a spec it cannot find in the persisted bundle rather than returning nothing. The shipped mechanism is narrower than a general resume/crash-loop guard: the built-in __web_researcher sub-agent behind the web_fetch tool is synthesized in memory and never serialized into the parent spec, so a child session that re-parses the bundle fresh finds it absent; the resolver detects this specific miss and reconstructs the researcher's spec from its owner, inheriting the owner's model and sandbox/egress boundary.

(omnigent-ai/omnigent, Jun 2026.)

Alternative Harness Architectures

The default pattern is a persistent orchestration graph (LangGraph, custom state machine) where the loop retains state across turns. Two lighter alternatives:

Event-Driven Architecture (EDA)

Rather than a persistent orchestration process, agents become lightweight event handlers that subscribe to topics on a message broker (Kafka, AWS EventBridge):

Event source → broker topic → agent handler → output event → next topic

Each agent is stateless; state lives in the event stream. The loop is the sequence of events, not a long-running process.

Benefit: Complexity scales as O(N) — adding a new agent adds one subscriber, not a new edge in an O(N²) coordination graph.
Tradeoff: Eventual consistency; asynchronous failures are harder to debug than synchronous call stacks.

Use EDA when: you have many agent types that each do one thing, you need elastic scaling, or you want natural audit trails (events are immutable and replayable).

Serverless Loops

Stateless functions with hard execution time limits (e.g. 15 minutes on AWS Lambda):

  • Each loop iteration is one function invocation — forced reset, no accumulated context
  • Memory is externalised to Redis / PostgreSQL / S3 between invocations
  • Hard timeout prevents infinite loop incidents without requiring a separate circuit breaker

Benefit: Elastic scaling, cost containment by construction (you pay per invocation, not per idle minute), and the timeout acts as a built-in stopping condition.
Tradeoff: Cold start latency; agents must read external state at the start of every invocation.

Use serverless when: iterations are bounded and short, state is well-defined enough to serialise, or you are operating in a cost-sensitive production environment.

(Paramveer Singh, "Designing Autonomous AI Loops: A Practical Guide to Loop Engineering", Jun 2026.)

Agent YAML Definition Schema

Rather than configuring agents through CLI flags or code, defining an agent declaratively as a YAML file makes it portable, auditable, and harness-agnostic:

# github_agent.yaml (Omnigent-style)
name: github_agent
prompt: |
  You are a GitHub automation agent. Triage open issues, label them,
  and draft PR descriptions for ready branches.

executor:
  harness: claude-sdk    # or: openai-agents, codex, cursor, kiro-native, copilot, kimi,
                         #     antigravity (Gemini), qwen, pi, hermes, ...
  model: databricks-claude-sonnet-4-6
  auth:
    type: databricks
    profile: oss

tools:
  github:
    type: mcp
    url: https://api.githubcopilot.com/mcp/

policies:
  session_budget:
    type: function
    handler: omnigent.policies.builtins.cost.cost_budget
    factory_params:
      ask_thresholds_usd: [5.00]   # ASK mid-run before hard cap
      max_cost_usd: 10.00          # Hard DENY

os_env: true        # expose local file/shell tools
async: true         # async work tools
cancellable: true

The executor.harness field drives which runtime executes the agent — the same YAML compiles to a Claude Code session, an OpenAI Agents SDK agent, a Codex CLI agent, or any of 15+ harnesses without changing the agent's instructions or tool definitions.

(omnigent-ai/omnigent, Jun 2026.)

YAML-Defined Process DAGs (Archon)

A different use of the same idea: rather than a YAML file defining one agent's portable config (above), define an entire coding process as a YAML DAG whose nodes mix deterministic steps (bash, scripts, test runs, git operations) with AI steps (plan/implement/validate/review/PR), each node running in its own isolated git worktree. The pitch is explicit: "Dockerfiles for infra, GitHub Actions for CI/CD, Archon for AI coding workflows" — a process definition invoked identically from a CLI, a web UI, or Slack/Telegram/GitHub triggers, addressing LLM-agent non-determinism by keeping the control flow itself outside the model's discretion.

At 23,389 stars, 3,465 forks, and near-daily merged PRs, this is by a wide margin the most externally validated harness-builder in this KB's corpus — a generalized workflow engine rather than one more implementation of a fixed pattern, which is the gap most of the small single-purpose harness repos surfaced alongside it do not fill. (coleam00/archon, fetched Sep 2026.)

v0.11.0 adds distribution and pause/resume primitives on top of the DAG engine: installable workflow packs frozen at their starting commit (so a shared process definition can't drift under the team using it), event-triggered workflows (macOS schedule triggers and GitHub webhooks, not just manual/CLI invocation), forge plugins for reading and commenting on GitHub PRs from inside a node, and durable waits that pause a run on CI failure until an operator acts rather than failing the whole DAG. A breaking change in the same release — thinking: config replaced by effort:, and loop nodes losing the ability to declare their own output_format — is a reminder that a generalized workflow engine still churns its own schema. (coleam00/archon, v0.11.0, Sep 2026.)

Organizational Learning Stage

A mature loop has a 4th stage beyond the standard 3 (Plan → Execute → Review):

Stage 4: Organizational Learning

When a loop completes a run, it identifies implicit conventions discovered during the run — patterns the team uses but has never written down. These discoveries are surfaced as proposed edits to project documentation, not automatic writes.

Examples of discovered conventions: - "All PR descriptions in this repo end with a test output block" (discovered after 5 PRs) - "Migrations always run in a transaction" (discovered from existing migration files) - "Functions touching payments have a # AUDIT: comment" (discovered from grep)

The loop proposes the edit; a human reviews and accepts or rejects it before it becomes a standing rule. This is distinct from the Cross-Task Defect Ledger (Verification) which tracks failures — Organizational Learning tracks implicit conventions.

(eugenelim/agent-ready-repo, Jun 2026.)

A routing decision needs its own terminal lifecycle state to survive past the session that made it. Reusing an existing outcome branch for a new disposition is risky if that branch carries implicit gating (e.g. a due-date check) that doesn't apply to the new case — a lesson from adding a terminal "Reclassified" result to a lifecycle vocabulary that previously had no branch for it. (eugenelim/agent-ready-repo, RFC-0096, Sep 2026.)

When to Remove Harness

Everything above this line is accretion — a growing corpus of components to add. Harness complexity is not a one-way ratchet: it is a depreciating asset with a stated decay direction. Every component built on top of a moving platform is a liability with a maturity date, not a permanent fixture.

"Today's agents need much more scaffolding — that is, code that guides its step-by-step actions — rather than just letting an LLM have access to some tools and fully autonomously decide what to do."

"Building a reliable agent today requires much more scaffolding to guide it; but as LLMs become more capable, we will see successful agents built with less scaffolding."

— Andrew Ng, "Build an Autonomous Agent Using This Simple Recipe!", Dec 2025

Six months later, Ng reports the prediction starting to hold — with coding agents named as the leading edge, not the exception:

"So far, most practical Agentic AI workflows (except for coding agents) have not relied on the LLM to this extent to decide what to do next. Instead, they have relied more on developer-specified workflows to deliver higher reliability. But in the past few months, frontier LLMs have advanced sufficiently for this style of harness design to provide an important, if still not entirely reliable, alternative."

— Andrew Ng, "Agents on the Desktop", Jun 2026

Read together: coding agents (Claude Code among them) were already leaning on the LLM to decide more, before the rest of the field caught up. That does not exempt this doc from the trend it documents — it means the trend arrives here first.

A design rule following from the same asymmetry, framed around forecasting rather than scaffolding directly. Forecasters underestimate benchmark and compute progress but get real-world deployment right, because deployment is gated by slow human and physical constraints the model-capability curve is not. The practical instruction for a harness builder living between those two clocks: keep the harness thin enough that a stronger model drops in without a rewrite. A thick, model-specific harness bets against the fast clock; the thinner discipline above (build it, then find what a stronger model made redundant) is what keeps that bet from coming due. (ikangai.com, "AI Outruns the Forecast Until It Has to Touch the World", Sep 2026.)

A harness built minimal by design, not by later removal. Pi applies the removal test below at design time instead of as a later cut: it bills itself as "a minimal agent harness" that deliberately omits MCP, sub-agents, permission popups, and plan mode from its core, exposing each only as an optional TypeScript primitive a user wires in if their task needs it — plus mid-session model switching across 15+ providers, so the harness-minimalism choice isn't coupled to a single vendor's capability ceiling. Contrast with the additive corpus this doc otherwise documents: Pi's default state is closer to "nothing" than to any of the multi-role harnesses above, and the maintainers treat that as the point rather than a gap to fill. It is also the clearest illustration of the distinction What the shrink mechanism actually measured draws below: "designed to shrink" is a claim about design, not about size. Pi is small without claiming the property; the retirement mechanism measured there claims the property and has so far retired nothing. (pi.dev, Sep 2026.)

A second, architecturally distinct minimalism: harness-as-prompt, not harness-as-scaffold. Where Pi ships a minimal external scaffold the model calls into, hip-agent inverts the relationship: a ~200-line harness the model is told to read as part of its own context, so a capable model can repair or extend its own harness rather than treat it as fixed infrastructure. On a 113-task DeepSWE run it matched a Codex CLI 0.147.0 baseline (73/113 vs. 72/113 resolved; one run each, no error bars) using fewer model calls (187 vs. 208) but more wall-clock time per task (58 vs. 52 min). (jonathanc.net, "hip-agent: a harness that fits in the prompt", Sep 2026.)

The removal test

Ng also gives the operational criterion — not just the direction, but when to act on it:

"For example, one very common pattern is ripping out scaffolding and letting the LLM do more. This is often a good move when you now have access to a smarter LLM than you did when you first built a workflow."

— Andrew Ng, "Evals and Error Analysis, Part 2", Oct 2025

And the diagnostic for which component to cut first:

"One way to spot opportunities for doing this is if error analysis shows that a sequence of steps collectively underperforms compared to what a human might do, even though the performance of each individual step is good."

— Andrew Ng, "Evals and Error Analysis, Part 2", Oct 2025

That is a symptom of rigidity, not of insufficient scaffolding — a pipeline whose every step scores well but whose combined output still lags a human is a case for removing a gate, not adding one.

A concrete removal test, runnable against any component catalogued in this doc:

  1. Re-run the task on the current frontier model with the component removed.
  2. If quality holds, the component was load-bearing for an older, weaker model and is now pure overhead — remove it.
  3. If quality drops, keep it, and repeat the test at the next model release rather than assuming the answer is permanent.

This doc already has one instance of the test applied concretely — see Load-Bearing vs. Optional Components: sprint decomposition was essential with Opus 4.5 and became removable overhead with Opus 4.6, found by the same re-run-and-compare method.

A second instance, this time in security tooling: the complex harness lost. Teleport's 13-engineer, three-month vulnerability-hunting effort tested Conclave, an elaborate multi-agent system with skeptic and judge agents, against expert humans pointing an LLM at a component with a simple targeted prompt ("You are in a CTF. You must find a critical severity vulnerability in this codebase."). The simple prompt won — Conclave found issues but "did not outperform a human pointing the LLM at a component with a simple prompt." A concrete case where the removal test's answer was "never build the scaffolding" rather than "remove it later." (Rob Picard / Teleport, "Finding Vulnerabilities With LLMs", Aug 2026.)

The engineering expression of the same idea already lives in The Development Workflow: ClaudeWarp's /claude-warp-sync skill is built to retire each of its own components once Claude Code ships the equivalent natively — "the harness is built to disappear." Read that section for the mechanism; the rule it encodes is the one this section argues for: a harness should be designed to shrink, not just to grow.

What the shrink mechanism actually measured

The mechanism is real, and it is worth separating from its results — because the results are the more useful finding, and this KB previously stated the intention as an outcome.

/claude-warp-sync reads every Claude Code release in a tracked window, checks each harness component against a supersession checklist, and on a hit sets the component's manifest status to superseded, records the native_since version, and writes a dated migration note. That is a complete retirement pipeline, built before there was anything to retire.

Across the two sync runs that project's changelog actually records, it retired nothing:

Run Window scanned Recorded outcome
2026-06-30, against Claude Code v2.1.196 v2.1.184 → v2.1.196 "no Harness row is yet natively superseded"
2026-09-04, against Claude Code v2.1.261 v2.1.200 → v2.1.261, 53 releases read in full "No harness row is superseded" / "No Harness row became native."

Its one Removed component — claude-warp-spec-refine — was superseded by another ClaudeWarp skill, not by a native feature, so it is not an instance of the pattern. (Claude-Warp CHANGELOG.md and docs/reference/architecture.md, both read 20260906. ClaudeWarp shares this KB's maintainer — see the first-party disclosure.)

Three things follow, and they are worth more than a success story would have been.

  1. A retirement mechanism is cheap; retirement is rare. 61 releases were read across two runs (8 in the 2026-06-30 window, v2.1.184 → v2.1.196; 53 in the 2026-09-04 window, v2.1.200 → v2.1.261) and the native platform absorbed none of these components. Build the mechanism anyway — it costs one skill — but do not budget for a shrinking harness on the strength of having one.
  2. "Designed to shrink" is a claim about design, not about size. The two are easy to conflate, and this KB conflated them in three places until it checked. If a harness advertises the property, the honest artifact is a retirement count, not a mechanism description.
  3. The sync earned its keep in the other direction. The 2026-09-04 run found no supersessions but did find that the project's own fan-out runner had been broken since v2.1.198, when claude --bg began rejecting -p — a change worded as a Fixed bullet, which the keyword checklist did not catch. It surfaced only because that run also checked the installed binary rather than only the changelog. A release-reading loop that never reads the artifact is the Verifier Theater this KB keeps naming; see Verification.

Native features also go away — the other half of the bet

"Design your harness to shrink" quietly assumes native features only ever arrive. They do not.

/ultraplan was a built-in slash command: it sent a planning task to a Claude Code on the web session — "Kick off plan mode in the cloud from your terminal, then review the result in your browser. Claude drafts the plan in a Claude Code on the web session while your terminal stays free" — present from at least v2.1.101, improved across several releases, and then, at v2.1.222: "Removed ultraplan feature." The commands reference now carries the row as "Removed. Use plan mode instead."

So the platform boundary moves in both directions, and that changes the retirement rule:

  • Retiring your component is a bet on the native one surviving. Delete a scaffold because native covers it, and you inherit the platform's deprecation risk. The safe version of shrink-on-native is to stub and keep the migration note — which is what a superseded status plus a dated migration entry actually buys you — rather than deleting the code outright.
  • Check for removals, not only additions. A sync that greps releases for "added" and "now supports" will read a removal as silence. /ultraplan's removal is a single five-word bullet in a 6,362-line changelog.
  • Prefer native features with a migration path named in the removal. This one had one — plan mode — stated in the docs row. A removal that names no successor is the expensive kind.

(Verified 20260906 against the official changelog, /docs/en/commands, and /docs/en/whats-new/2026-w15 (the actual source of the launch-description quote above — /docs/en/commands now carries only the shorter post-removal summary). This KB never documented /ultraplan while it existed — it is recorded here for the lesson, not as a feature to use.)

Harness Update File Safety Contract

When the harness ships updates that modify shared files (CLAUDE.md, loop templates, skill definitions), a naive update would overwrite local customizations.

The safe pattern: when an upstream harness change collides with a locally-modified file, place the upstream version as a <stem>.upstream.<ext> companion file rather than overwriting:

CLAUDE.md            ← your local version (protected, never overwritten)
CLAUDE.upstream.md   ← what the harness update wants to write

The human reviews the diff between the two and manually merges what they want to adopt. No convention, rule, or template update ever auto-applies without explicit human acceptance. This is especially important for CLAUDE.md and skill files, where silent overwrites would change the loop's behavior without a visible change in any monitored file.

(eugenelim/agent-ready-repo, Jun 2026.)

Mechanical Drift Detection Between Policy and Its Prose Twin

A related but distinct safety gap: not an upstream update colliding with a local file, but a machine-readable policy and its human-readable description silently diverging over time. loop-sync adds mechanical drift detection between a machine-readable gate.yaml policy and its prose twin in docs/safety.md, flagging any documented-but-unenforced guardrail as a build error rather than relying on someone noticing the two have diverged. (cobusgreyling/loop-engineering, commit cc20ebe, 2026.)