Fleet Engineering¶
When you have more agents than you can watch, you need fleet ops.
What it is¶
Fleet engineering is the discipline of managing multiple AI agents at enterprise scale — not designing individual loops, but operating the infrastructure that governs, observes, and routes work across a pool of them.
Coined (or popularised) by Cobus Greyling (Jun 2026).
How it differs from loop engineering¶
| Loop engineering | Fleet engineering |
|---|---|
| Designing a single loop (trigger → act → verify → state) | Operating many loops across an organisation |
| Concern: stopping conditions, guard scripts, state files | Concern: agent health, cost governance, audit trails, role assignment |
| Unit of design: one loop | Unit of design: the fleet |
| Audience: individual engineers | Audience: platform teams, AI ops |
The "audience" row is a default, not a hard boundary — it can become an organization's default bottom-up. OpenAI's Sherwin Wu, who leads engineering for its API platform, reports "roughly 95% of engineers use Codex, often working with fleets of 10 to 20 parallel AI agents" — an org-wide norm among his individual engineers, not a platform team's centrally governed fleet. Read as loop engineering scaled by adoption rather than fleet engineering by design: the same episode credits an unnamed process change with cutting code review turnaround from 10-15 minutes to 2-3 minutes, though the mechanism sits behind a paywall and is not independently confirmed here. (Lenny's Podcast, "Engineers are becoming sorcerers" — Sherwin Wu, Feb 2026.)
Core concerns¶
1. Governance Which agents are authorised to run, with what tools, spending limits, and data access? Fleet engineering answers these questions at the organisation level, not per-loop.
2. Observability - Are agents completing their runs? With what success rate? - Which agents are consuming the most tokens? - Where are loops failing silently?
Tools like LangSmith expose agent-level telemetry across all running loops. Comet's Opik (open-source) adds continuous LLM-quality scoring: Online Evaluation Rules run a judge model against every agent output in production, alerting when answer relevance or hallucination scores fall below threshold. This is the fleet equivalent of a test suite — watching quality across the whole fleet, not just per-run exits.
3. Routing and role assignment A fleet may have specialist agents (security reviewer, dependency updater, test writer). Fleet ops decides which work goes to which agent — a higher-level scheduling problem than a single loop's internal orchestration.
4. Failure propagation In a fleet, a misconfigured prompt or runaway loop can cascade across many agents before anyone notices. Fleet engineering adds circuit breakers, spend caps, and automated rollback.
5. Shared config/credential concurrency
Two failure classes have shipped from letting multiple Claude Code instances share one ~/.claude config/credentials store at once — worth naming directly against this doc's own "what happens with 50 running simultaneously" question.
Config corruption. As of v2.1.59, concurrent instances could corrupt the config file badly enough to wipe authentication ("Fixed config file corruption that could wipe authentication when multiple Claude Code instances ran simultaneously") — one entry in a longer-running history of config-corruption fixes (atomic writes, a single backup file, concurrent-write corruption on Windows) spanning many versions. As of v2.1.259, concurrent sessions also no longer silently revert each other's ~/.claude.json changes (workspace trust resetting, MCP/project state lost). Treat this as a recurring risk class, not a bug closed once.
MCP OAuth token refresh races. A separate v2.1.59 fix addressed "MCP OAuth token refresh race condition when running multiple Claude Code instances simultaneously," but the family recurred both before and after — a macOS keychain overwrite race, a stale cached-token read race, and, most recently, a v2.1.221 fix for "a rare wake-from-sleep race where two Claude Code processes could both refresh the same MCP connector or WIF OAuth token at once, forcing re-authentication."
Mitigation. Give each fleet worker its own CLAUDE_CONFIG_DIR rather than sharing one ~/.claude across concurrent processes. Per Anthropic's Authentication docs, this isolates credentials completely, including on macOS: "If you've set the CLAUDE_CONFIG_DIR environment variable, Claude Code keeps the .credentials.json file under that directory instead, including the file the macOS fallback writes, and keys the macOS Keychain entry to that directory too, so a session with a different CLAUDE_CONFIG_DIR reads a different entry." If a fleet still sees unexpected /login prompts or a re-authentication storm after isolating CLAUDE_CONFIG_DIR per worker, check the current version's changelog for OAuth-race fixes before assuming misconfiguration.
(Anthropic, CHANGELOG.md, v2.1.59, v2.1.221, v2.1.259; Authentication, checked 2026-09-07.)
6. Storage architecture that was fine at F0/F1 breaks silently at F2+. A concrete production lesson, reported as "the biggest design mistake" in a persistent-agent platform's own build: synchronous SQLite storage was fine for one agent reporting to one user over Slack/iMessage, but broke down once a single agent instance was fielding 50 parallel sessions with a whole team on it. The failure mode is the same shape as the config/credential concurrency risk above — a storage choice that is invisible at low concurrency and becomes the bottleneck exactly when a fleet crosses from F1 into F2+ scale — but at the application's own data layer rather than Claude Code's credential store. (@steipete, Sep 2026.)
The fix is running as one long-lived agent goal rather than a rewrite: "I have a /goal with Astra that so far landed 575 PRs to move everything to to [sic] async workers. We ship these improvements as we make progress." By Sep 26 it had run "15 days so far" (@steipete). Asked whether a rewrite had crossed his mind: "Yeah but we just refactor in place. Less chance for things to break. Base architecture is good." (@steipete). Asked how to test a refactor that size: "We shipped 3 releases while it slowly refactors the codebase." (@steipete). The remediation for a fleet-scale storage mistake was an incremental, continuously shipped refactor under one goal, not a stop-the-world migration. (@steipete, Sep 2026.)
Relationship to loop engineering¶
Fleet engineering is above loop engineering in the stack:
Fleet engineering ← routes, governs, observes N loops
└── Loop engineering ← designs, schedules, guards 1 loop
└── Agent ← a single Claude invocation
A practitioner who has mastered loop engineering designs excellent individual loops. A fleet engineer asks: "What happens when we have 50 of these running simultaneously?"
Fleet Four Pillars¶
Cobus Greyling (Jun 2026) defines fleet engineering around four operational concerns:
| Pillar | What it governs |
|---|---|
| Delegate | Which tasks flow to which agents — a routing and scheduling problem above individual loop design |
| Improve | Continuous tuning of agent prompts and harnesses based on fleet-level telemetry |
| Approve | Human-in-the-loop gates for high-consequence actions across the fleet |
| Connect | Integration layer — MCP connectors, webhooks, APIs that wire agents into live organisational systems |
Fleet Four Maturity Levels (F0–F3)¶
Distinct from developer maturity (docs/20) and per-loop readiness (L1/L2/L3 in docs/34):
| Level | Description | Gate to advance |
|---|---|---|
| F0 | Ad-hoc: individual loops run manually with no coordination | At least 3 loops running reliably at L2+ independently |
| F1 | Coordinated: loops share a STATE.md registry; no agent claims conflicting tasks | Zero collision incidents in 2 weeks; acting_on field enforced (see Loop Patterns) |
| F2 | Governed: Fleet budget guard active; spend visible per agent; approval gates for irreversible actions | Fleet budget never exceeded; audit log in place |
| F3 | Autonomous: fleet self-governs; anomaly detection triggers automatic rollback; humans review summaries, not individual runs | No undetected runaway spend in 30 days; full observability coverage |
Start every fleet at F0, regardless of engineer experience. F3 requires F2 evidence.
Observable per-gate indicators — filled 2026-09-06¶
This section carried an open gap for two months: the levels above describe postures, and a posture is not a checkable signal. Three sources were checked and ruled out (thirai-classlab/hirai-method, chf3198/megingjord-harness, ruvnet/ruflo — each documenting infrastructure a passing fleet would have, not a test that it passed). The example worth keeping: a governance-first harness enforcing shared policy across Copilot, Claude Code and Codex via a "baton workflow" (Manager → Collaborator → Admin → Consultant) documents fleet-aware routing, telemetry and policy enforcement consistent with F0–F3 progression — and still names no gate signal. (chf3198/megingjord-harness, Jul 2026.)
A second cross-vendor example widens the roster further — Claude Code, Codex, Grok, Kimi, DeepSeek Harness, OpenCode, and Pi, coordinated through a uniform MCP tool layer rather than a shared control plane — and comes closer to an F2 gate signal without quite naming one: pre-agent policy hooks gate delegation with per-harness quotas, and a live cost ledger tracks spend across all seven vendors in one place. That is closer to the F2 "spend visible per agent" criterion than either source above gets, but it is still infrastructure a passing fleet would have, not a test that it passed — the gap this section closed stays closed by cobusgreyling/fleet-engineering below, not by this repo. (firstintent/ccteam, 425★, Sep 2026.)
The answer was inside a repo this doc already cites.
cobusgreyling/fleet-engineering — referenced
three times here — carries docs/maturity-model.md and docs/fleet-ready-score.md, which nobody
had opened. They give exactly the missing thing: file-existence checklists per transition, plus
a numeric threshold from a runnable auditor.
Two different F0–F3 scales — do not merge them
The table above is this KB's own fleet scale, gated on loop-behaviour evidence (3 loops at L2+, zero collisions in 2 weeks, no undetected runaway spend in 30 days). The scale below is cobusgreyling's, gated on artifacts existing in the repo. They share four labels and otherwise share nothing — different criteria, different authors, different units. Treat the one below as an instrument you can run today, and the one above as the outcome it is proxying for. An artifact checklist is cheap and mechanical; behavioural evidence is what actually proves a gate. Use both, and never quote a level without saying which scale it came from.
Its exit criteria, verbatim:
| Transition | Criteria |
|---|---|
| F0 → F1 | FLEET-STATE.md or registry file exists · each production-touching agent has an owner · accountability test attempted for one recent action |
| F1 → F2 | 100% of active agents in registry · identity model documented · audit path exists (traces or structured logs) · fleet-audit score ≥ 40 |
| F2 → F3 | budget alerts fire before hard stop · inbox SLA for risky actions · incident runbook references accountability test · fleet-audit score ≥ 65 |
The score comes from npx @cobusgreyling/fleet-audit, banded F0 0–39 · F1 40–64 · F2 65–84 ·
F3 85–100, with --min-score=40 offered as the CI gate for "we know our agents" and 65 for a
shared team fleet.
Quote its own caveat with it, because it governs how the numbers may be used: "Exact numbers may drift; levels and the accountability sentence must not." So cite the bands and the checklists; do not build anything on the individual point weights.
Two things make this a genuine gap-closer rather than another posture description. The criteria are file-existence checks — mechanically decidable, no judgement — and the threshold is produced by code you can run, so "we are F2" becomes falsifiable. Its own mapping also lands on this KB's loop levels directly: "L3 unattended loops | Require F2 budgets + kill switch minimum."
The process lesson is the sharper one. This gap sat open through three targeted retries while
the answer was in a repo already cited three times in this very doc. Every retry searched for a
new source; none re-read the ones already trusted. Before searching outward for a gap, exhaust
the repos already in SOURCES.md — a citation is not a read.
(docs/maturity-model.md
and docs/fleet-ready-score.md,
fetched 20260906.)
Fleet Economics — Cost Attribution¶
Individual loop token budgets aggregate into fleet-level spend. At F2+, track cost per agent:
fleet-budget.md entries:
Daily Triage Loop: budget $0.50/day · actual $0.31/day · headroom 38%
PR Babysitter: budget $1.00/day · actual $0.67/day · headroom 33%
CI Sweeper: budget $0.20/run · actual $0.04/run · headroom 80% (early exit working)
Fleet total ceiling: $5.00/day
Cost attribution rule: if you cannot identify which agent generated a runaway cost event, you are below F2. Observability is a prerequisite for governance, not an enhancement.
(cobusgreyling/fleet-engineering — Cobus Greyling, Jun 2026.)
Claw vs. Assistant Identity Choice¶
At F1+, each agent in the fleet must have a declared identity:
| Identity | Behaviour | Use when |
|---|---|---|
| Claw (autonomous) | Takes action without per-action human approval; uses auto mode; emits structured audit log | Loop is well-tested at L2, action set is low-risk, fast turnaround required |
| Assistant (assisted) | Proposes actions for human approval before execution; uses ask mode | Loop is new (L1), action set is irreversible, or human oversight is required by policy |
Default all new fleet agents to Assistant until they reach L2 operational readiness. Promote to Claw only after the L2→L3 gate is passed and the action set is proven safe.
(cobusgreyling/fleet-engineering, Jun 2026.)
Org-Chart Coordination (an alternative to a graph topology)¶
Most fleet coordination is described as a graph — nodes (agents) and edges (handoffs). An alternative: map agents onto an org chart instead — each agent has a role and a reporting line, and agents coordinate the way a human org does, over email rather than a shared state file or direct API calls. The org-chart framing gives fleet designers a vocabulary non-engineers already understand (manager, report, cc'd stakeholder) and makes escalation paths explicit by construction — a report's blocked question routes to its manager the same way a human employee's would. Backend-agnostic: agents in the chart can run on Claude Code, Codex, or OpenCode interchangeably. (alookai/alook, Jul 2026.)
A third topology: swarm coordination via consensus protocols. Where the graph and org-chart framings coordinate through explicit messages or reporting lines, a swarm topology coordinates through a queen-led hierarchy backed by distributed-systems consensus (Raft, Byzantine fault tolerance, or gossip protocols) — the same class of protocol used to keep database replicas agreed on state, applied here to keep agent fleet state agreed across many concurrent workers. This trades the readability of an org chart for the fault-tolerance guarantees of a consensus algorithm — appropriate when the fleet must keep operating correctly through individual agent crashes, not just coordinate cleanly when everything is healthy. (ruvnet/ruflo, Jul 2026.)
A later ADR turns swarm start from a fire-and-forget dispatch into a steerable long-horizon run:
NDJSON progress events stream out while the swarm executes, and a file-based inbox lets an
operator inject mid-run guidance without restarting the swarm — a rare concrete answer to "how do
you steer a fleet that's already running" rather than only gating it before dispatch or reviewing
it after. (ruvnet/ruflo, ADR-385, Sep 2026.)
A fourth: runtime isolation that moves with trust. Below all three topologies above sits a question they each assume away — how tightly isolated two coordinating agents are at the infrastructure level. A bare-metal hypervisor with trust/communication-driven coherence domains answers it dynamically: agents that communicate more move closer together (relaxed isolation, lower latency); agents whose trust score drops split apart into harder isolation — the runtime infrastructure analogue to ruflo's Trust Loop scoring, but enforced at the hypervisor rather than the orchestration layer. Reports sub-millisecond isolation-boundary switching versus seconds for VM-based approaches — relevant wherever fleet trust decisions need to translate into an actual containment change, not just a routing decision. (ruvnet/rvm, Sep 2026.)
Mapping how the pieces of one ecosystem interoperate. As a fleet's supporting tooling grows (orchestration, harness scoring, runtime canary/rollback, memory, budget gates), the map of how those pieces compose becomes its own artifact worth documenting. ruClip is a README-level control-plane map of ruvnet's own stack — org-chart-style budget gates and approval-gate state machines composing ruflo (orchestration), metaharness (build-time scoring), autogenous (runtime canary/rollback, see Loop Contract), and RuVector+AgentDB (memory) — useful less for its own mechanism than as a worked example of documenting fleet-tooling composition explicitly rather than leaving it implicit across separate repos. (ruvnet/ruClip, Sep 2026.)
Naming a concrete scale threshold where coordination becomes the problem. One coordination-layer product frames "roughly twenty-plus agents" as the point organizational coordination — not agent capability — becomes the binding constraint, and answers it with atomic task checkout (preventing two agents from silently duplicating the same work), event-driven "heartbeat execution" instead of polling (echoing the polling-loop failure pattern in Common Failure Patterns), and hard-stop budget/token enforcement as the stopping condition. Consistent with this doc's own framing that fleet engineering is an organization-level problem distinct from any single loop's internals, but treat the specific "twenty-plus" threshold as one vendor's rule of thumb, not a measured inflection point. (explainx.ai, "Paperclip: Run a Company of AI Agents", Sep 2026.)
A fifth: let the model organize the group. Every topology above is hand-designed. Ethan Mollick argues he got the management model wrong: applying the Bitter Lesson to the org chart means models organize agent groups themselves. His evidence is an OpenAI swarm of thousands of agents that exchanged about 2.7 million messages over 88 hours with thin human coordination, and a Codex "Ultra" mode that spun up 3 sub-agents on its own, and 13 when he had sketched three teams (Mollick, "The Dot and the Swarm", Oct 2026). A response, "Beekeepers, not managers", accepts the premise and moves the human's job to boundary conditions, with four loop-design habits: mix models to avoid correlated-error monocultures, build an aggregate observation window, fence permissions, budgets and irreversible actions up front, and inspect at checkpoints while treating an agent's account of its own work as a claim to verify. A customer story on running 10,000 agents for 36+ hours (OpenAI, Third Moment Research) is a similar scale data point, but only its RSS title and description were readable (the page returned HTTP 403), so no mechanism is recorded here. Treat the swarm numbers as single-source and vendor-adjacent.
Control Plane Framing: Kubernetes for Agents¶
Several Oct 2026 pieces from the cloud-native world converge on one split: the agent reasons, and a platform owns hosting, identity, permissions, audit and execution.
| Source | The split it proposes | Maturity stated by the source |
|---|---|---|
| CNCF blog, "The case for a cloud native agent harness" (Stacklok, Sep 28 2026; summarised in The New Stack) | Desktop harnesses fuse UI, agent loop, sandbox, credentials, tools and session DB in one process; "a monolith in a container is still a monolith". Separate the agent loop from clients, execution environments, tool catalog and supporting services | Open-sourced Mecatl; identity (a SPIFFE trust domain with the delegation chain in a JWT), scoped resource grants and signed context are listed as design drafts, not shipped |
| OpenClaw Enterprise (OpenClaw blog, 2026-09-29; also covered by The New Stack) | A control plane with isolated namespaces, credentials, permissions and audit over persistent agents, with a swappable harness, model and sandbox | Pre-1.0, pilot only; the post states IT's default stance is to ban such agents |
| kagent and Karpenter, per The New Stack | A loop of observe, diagnose, propose, approve, reconcile, verify, with agents reasoning and deterministic operators executing | An article's worked example on a CNCF Sandbox project |
The one concrete durability claim is worth keeping exact: Mecatl keeps session state and event history in durable storage under single-writer coordination, so a replacement worker resumes from the last persisted turn boundary. The post says this does not resume an in-flight operation and work after the last save can be lost: turn-level durability, not a transaction. For a fleet that is the difference between a pod eviction ending a conversation and ending a unit of work, and it is the fleet-level sibling of Long-Running Agents checkpointing. The post also argues a hosted agent should start from a catalog of allowed tools rather than an unrestricted shell, which lines up with the Approve pillar above and with Permissions.
Shared Gateway for a Team: One Trust Domain, Routed Humans¶
The control-plane rows above put many agents behind one platform. OpenClaw's Team setup guide describes the opposite shape: many humans around one agent host. It sets up "one OpenClaw gateway that a whole team uses" and says "team operation is configuration, not a separate edition". (Vendor documentation, read in full 2026-10-19; the configuration was not run here.)
| Concern | What the page says |
|---|---|
| Trust | "A gateway is one trust domain." Everyone who can message a tool-enabled agent shares its delegated tool authority. Ownership, presence and roles are "collaboration guardrails inside the boundary, not isolation between adversaries"; mutually untrusted users or organisations get separate gateways |
| Access | Loopback bind by default. Teammates come in through Tailscale Serve (listed as recommended), an identity-aware proxy such as Cloudflare Access, or a shared token or password, which gives everyone one owner profile instead of per-person identity |
| Attribution | Each shared session has "an immutable creator, an assignable owner, and the history of people who actually prompted"; sessions are assigned "like GitHub issues". With Git co-author credit enabled, commits carry Co-authored-by trailers and generated pull requests link back to the session |
| Bounds | gateway.roles bind a profile to the sessions it can touch, the agents it can use, a ceiling of operator scopes and whether its new sessions must be sandboxed. The example gives guests view-only access to others' sessions and notes this still lets them read every shared session, "including tool output" |
The fleet lesson is the line the page draws: collaboration controls are not tenant isolation. The
nearest row above for the untrusted case is OpenClaw Enterprise (isolated namespaces, pilot only); the
Team page does not mention it, so that pairing is this KB's. One public example of the attribution
layer: all 13 commits of OpenClaw PR #161656
(merged 2026-09-30; the change is in
Session Architecture) are
authored by roboclaw-bot with a Co-authored-by trailer naming steipete, and the body lists
"@steipete" under "Worked on by" and links a "Work sessions" page (checked with gh api 2026-10-19;
the session was not opened). Consistent with the page; it does not show the configuration behind it.
Routing work between people and a shared agent. Two informal posts from @steipete, who built OpenClaw with the community (per its README), add the routing side; they are his own description, not documentation. On 2026-10-03 he explained a slide as "a joke around being the meat proxy": the team "moved to one agent, building together, everyone can see or prompt each others' sessions. Eliminating meatproxyism", linking the Team page (post). On 2026-10-07: "I hooked up our team claw to X to trigger work faster. Unassigned sessions are for anyone to grab. Our agent looks who worked on the related code last and pings people on the server. Whole thing was a prompt and team server extended itself since plugins are now hot reloadable." (post.) The Team page, read in full, covers ownership and assignment but not the X trigger, the last-toucher ping or hot-reloadable plugins, so those three rest on the post alone, which reports no result for any of them (event trigger: Background Agents).
Unattended Fleet Mechanics: Wake, Recovery and Human-Wait¶
A different set of Oct 2026 release notes, from small repos, converge on the failure modes of fleets that run without anyone watching. Each row is one pattern; the cells name the projects in this batch that ship it. All come from release notes or commit text, and several were read at title level only, so treat them as design signals, not verified behaviour.
| Pattern | Who ships it |
|---|---|
| De-duplicate wakes and launches | OpenRig 0.6.11: a parked seat is no longer re-woken for unchanged queue rows (and a known issue is documented where a held-back one-shot wake counts as the timer's single fire). omnigent v0.16.0: brief runner drops no longer fail turns and host launches no longer spawn duplicate runners. dream-machine #196: a scheduleContract with receipts and a duplicate-firing skip |
| Never let an automated trigger answer a human question | OpenRig 0.6.7: a send or watchdog wake is refused while the seat's latest hook says it is waiting on the human, and the wake goes up the escalation path. session-orchestrator ADR-0016 (rules CSM-001..005, v5.8.0): an incoming peer message is a claim to re-verify, no permission laundering, delivery never guaranteed |
| Make the human pause a queue, not an ending | Archon v0.12.0: one "attention queue" routes every human-in-the-loop pause, and public stages pause as an attention item instead of ending. OpenRig 0.6.11: restore stops for a decision rather than starting a seat fresh |
| Durable, resumable state | Archon v0.12.0: workflows can run without a database (store: files) and keep revisioned durable state; a run that hits a provider usage limit with a known reset pauses and auto-resumes without holding a resource slot. session-orchestrator v5.11.0: release verification resumable from a persisted receipt. Orchestra v2.46.0: per-worker durable checklists. agent-orchestrator v0.13.2: durable scheduled automations |
| Stop on stalls as well as on wall-clock | hearting v3.35.0: dispatched foreground workers get a second, earlier stop when they show no activity, with the hard timeout kept as the final backstop (PR #577; the release title alone reads as if it replaced blanket timeouts) |
| Unknown is not zero | session-orchestrator v5.11.0: absent usage is reported as NO-USAGE, a missing population as "not measurable", and overflow write failures are reported rather than swallowed. Same defect class as the one named in Failure Patterns |
| Enforce by mechanism, not prompt | hearting v3.12.0: OpenCode workers marked no-commit are refused git commit by the runtime (the release's fix is scoped to opencode). Orchestra v2.46.0: a shell guard, described as "a narrow policy, not a sandbox". session-orchestrator v5.6.0: a fail-closed wave-scope gate for parallel agent waves |
| Reject an unfinishable loop before it runs | OpenRig 0.6.8: validate refuses a step with no allowed exit that can finish it or route onward (step_cannot_finish). Archon v0.12.0: a when: that can never be evaluated now fails at load |
Two limits the rows share. First, a concurrency cap is a fleet control in its own right:
agent-orchestrator v0.13.2 lists project agents capped at 6 (one line in the notes; behaviour beyond that
is not verified). Second, the ADR-0016 measurements bound what a fleet can assume about coordination
inside Claude Code: peer sessions on the same host are discoverable and messageable via
ListAgents/SendMessage, sub-agents can send upward but not discover sideways, headless -p
workers can accept messages, and cross-host Remote Control was asymmetric on the version measured (the Mac saw the server, the server
saw nothing; delivery was measured in one direction only; a research preview). See Harness Patterns for the same projects' dispatch
and receipt designs, and Common Failure Patterns for the polling and false-success
classes several of these fixes answer.
Case Study: Gas Town — 20-30 Parallel Instances via Git-Persisted Work Units¶
Steve Yegge's "Gas Town" is a concrete, named production deployment of fleet engineering at real scale: 20–30 parallel Claude Code instances coordinated through work units called Beads, persisted in git rather than in an external queue or database. Two mechanisms make this survive at that concurrency:
- Merge-queue management — Beads flow through an explicit merge queue rather than each agent pushing directly to a shared branch, turning "many agents editing the same repo" from a collision risk into a serialized, auditable sequence.
- Crash-surviving agent identities — an agent's identity and in-progress work are anchored to git state, not to a live process; a crashed instance can be restarted and resumes against the same Bead rather than losing its claim or duplicating another instance's work.
This is a working example of the Fleet Budget Guard and claimed-todo ideas at a concurrency level (20–30 agents) well beyond what most documented fleets in this KB report operating at. (Steve Yegge, "Welcome to Gas Town", Jul 2026.)
Case Study: Bun — 64 Parallel Instances Rewriting 535K Lines in 11 Days¶
Bun's port of its JavaScript-runtime internals from Zig to Rust is a second concrete, named production deployment at real scale, larger in raw instance count than Gas Town: about 50 dynamic workflows over 11 days, peaking at 64 parallel Claude Code instances across 4 git worktrees — "4 of these workflows at once each in a separate worktree, each with 16 Claudes per workflow. About 64 Claudes at a time" (Bun, "Bun, in Rust") — sustaining roughly 1,300 lines of code written per minute at peak. Unlike Gas Town's Beads-based coordination layer, this fleet ran on plain git worktrees with no external work-unit queue — a data point for how far raw worktree parallelism scales before a coordination layer becomes necessary.
What "535,496 lines" actually measures¶
535,496 is the source Zig codebase, not a size of the rewrite or its output — and this doc previously stated the figure without saying which of three legitimate, independently-sourced numbers it was, which is exactly why a critic flagged it as contradicting the ~750,000 figure quoted elsewhere. It doesn't; the three numbers measure three different things:
| Number | What it measures | Source |
|---|---|---|
| 535,496 lines | Lines of Zig in the source codebase, excluding comments, across 1,448 .zig files |
Bun, "Bun, in Rust" |
| ~750,000 lines | Anthropic's rounded count of the resulting Rust codebase | Anthropic, "Introducing dynamic workflows in Claude Code" |
| 1,009,272 lines | Lines added in the final merged diff — "the diff that landed was +1,009,272" (Bun, "Bun, in Rust") — includes churn, not just net-new code | Bun, "Bun, in Rust" |
The general lesson generalises past this one case study: a line count (or any figure) without its unit and its population is a claim waiting to be flagged as an error by the next reader who finds a different, equally correct number for the same-sounding thing.
Two design choices carried the fleet through the port:
- A checklist-shaped task, not a single instruction. Fixing the cyclical dependencies "revealed about 16,000 compiler errors. A massive number for 1 human, but not a crazy number for 64 claudes at once" (Bun, "Bun, in Rust") — treated as a distributable checklist and spread across the 64 parallel instances — see the granular decomposition sequence in The Factory Model.
- Blind adversarial review at fleet scale — Bun describes the ratio as "1 implementer, 2 or more adversarial reviewers per implementer. The reviewer's only job: find bugs & reasons why the code does not work" (Bun, "Bun, in Rust"); Anthropic separately describes the same review layer as "hundreds of agents working in parallel with two reviewers on each file" (Anthropic, "Introducing dynamic workflows in Claude Code"). It produced 128 bug fixes in the v1.4.0 release against only 19 regressions introduced across the full rewrite.
Result: 6,502 non-merge commits (6,778 including merges) over eleven days from first commit to merge, with 99.8% of the existing test suite passing on the resulting Rust codebase (Anthropic, "Introducing dynamic workflows in Claude Code"), 128 bugs fixed vs. the prior release, and memory leaks eliminated (one multi-build leak dropped from 6.7 GB to 609 MB over 2,000 iterations) — the class of bug the rewrite was undertaken to fix in the first place. (Bun, "Bun, in Rust", Jul 2026; Anthropic, "Introducing dynamic workflows in Claude Code", May 2026.)
Counterweight: an uncritical case study deserves its objections stated¶
Every figure above comes from the company that ran the fleet and wrote it up (Bun) or the vendor that sold the platform it ran on (Anthropic) — this doc has not, until now, carried anything from outside either party. The Hacker News discussion of Anthropic's launch post raises two objections worth reading alongside the results:
The result may not be specific to dynamic workflows. A behaviour-preserving, file-by-file port is close to a best case for coding agents regardless of the orchestration framework running them:
"Mechanical refactors are relatively straight forward for agents."
If that reading holds, the case study is stronger evidence that fleet parallelism scales to 64 concurrent instances than it is evidence that dynamic workflows specifically — as opposed to worktrees plus adversarial review, which Gas Town also uses without dynamic workflows — caused the result.
Token cost for the Bun port is published, and it is large. Bun's own post gives the whole-port figure verbatim: "Pre-merge, this took 5.9 billion uncached input tokens, 690 million output tokens, and 72 billion cached input token reads — around $165,000 at API pricing." Read the scope before reusing the number: it covers the entire eleven-day, ~50-workflow port up to merge, not one 64-instance wave. The post publishes no per-run or per-instance breakdown, so the figure bounds the whole effort rather than pricing a single fleet.
Which rate card — and what the same work would cost on another. The post names the model: "I used a pre-release version of Claude Fable 5, a Mythos-class model." Repricing its own published token counts against Fable 5's rates ($10 input / $50 output / $1.00 cache read per MTok) gives $165,500 — within 0.3% of the $165,000 quoted, which is what makes the Fable-5 basis legible rather than assumed. The same tokens on the other current tiers (our arithmetic, not Bun's):
| Rate card | Same 5.9B in / 690M out / 72B cached reads |
|---|---|
| Fable 5 — $10 / $50 / $1.00 (what was used) | ~$165,500 |
| Fable 5.1 — $10 / $50 / $0.25 | ~$111,500 |
| Opus 5 — $5 / $25 / $0.50 | ~$82,750 |
| Sonnet 5 — $2 / $10 / $0.20 | ~$33,100 |
The cached reads dominate. 72 billion of them at Fable 5's $1.00/MTok is $72,000 — 44% of the bill, more than input and output combined. Fable 5.1 reads at 4x cheaper for the same $10/$50 base, so the identical run costs ~$54,000 less with no change to the harness. In a fleet this shape the lever is the cache-read price, not the output price — see Cost & Turn Control.
Per-seat plan economics are a separate question, and there the reports are of hitting limits fast. Other users of the same feature, on smaller, unrelated jobs:
"Tested this out on a 5x max plan, turns out I spun up 62 Opus 4.8 1M sub-agents for my dynamic workflow and maxed out my ~5hr cap in..... 18 minutes?"
— Syntaf, comment on Hacker News item 48311705
"I just hit my Claude Max limit for the first time ever thanks to workflows lol. Like 90 agents ran to do a code review of a fairly small package I have."
Read the throughput and bug-fix numbers above as real but vendor-reported and favourable-case; treat the cost of reproducing this at similar concurrency as unstated by either primary source and plausibly large, per what practitioners report when they run the same feature at a fraction of the scale.
Case Study: GitHub Copilot's Own Rust Rewrite — a Different Playbook¶
A third named large-scale agent-driven rewrite, contrasted directly against the Bun case study above by the outlet that covered both: GitHub rewrote its own Copilot agent runtime (800,000+ lines) primarily with its own Copilot coding agents across 128 incrementally-merged PRs — an incremental, PR-gated approach rather than Bun's single eleven-day, ~50-workflow push. Read alongside Bun, the comparison is evidence that "coding agents can drive a large rewrite" generalizes across at least two independently-run playbooks, not just one company's specific process. (The New Stack, "GitHub and Anthropic used their own agents for major Rust rewrites — but with very different playbooks", Sep 2026.)
Pattern: Desktop Multi-Agent Office (a "GOD agent" coordinator)¶
A distinct topology from the org-chart and swarm-consensus patterns above: a desktop application runs several real vendor CLI agents (Claude Code, Codex, Grok — 12 distinct CLIs in the reference implementation) as actual terminal processes inside an "office" metaphor, coordinated by one privileged "GOD agent" that routes incoming tasks to the right worker process and escalates only spend, destructive-operation, or scope decisions back to the human — every routine decision stays inside the office. This is the Hierarchical Delegation pattern from fleet-engineering made concrete at the desktop scale rather than the org scale: one coordinator, several heterogeneous worker processes, a narrow human-escalation channel. Notable primarily for external validation at a scale none of this doc's other fleet reference implementations have reached: 6,448 stars, the #1 GitHub Trending Repo of the Day on its release. (chaitanyagiri/munder-difflin, Sep 2026.)
Current state (June 2026)¶
- cobusgreyling/fleet-engineering is the primary reference implementation with six production patterns (Team Registry, Shared Inbox HITL, Hierarchical Delegation, Fleet Budget Guard, Cross-Agent Audit)
- LangSmith is the leading commercial observability platform
- Most teams are still at F0–F1; F2+ governance is rare in the wild
- The term is new; expect terminology to stabilise as the discipline matures
See also¶
- The Six Building Blocks — the foundation for individual loops
- Learned Orchestration — training the orchestrator vs. coding it
- Fan-Out — parallelising at the loop level (a step toward fleet thinking)
- Cost & Turn Control — mandatory at fleet scale