Fan-Out — Parallelizing at Scale¶
For large migrations, analyses, or batch operations, distribute work across many parallel Claude invocations:
# Step 1: Generate task list
claude -p "list all Python files that need migrating to async. output one path per line" \
> files_to_migrate.txt
# Step 2: Fan out across files
while IFS= read -r file; do
claude -p "Migrate $file to async/await. Preserve all behavior. Return OK or FAIL." \
--permission-mode auto \
--allowedTools "Read,Edit,Bash(git commit *)" \
--max-turns 15 &
done < files_to_migrate.txt
wait
Test on 2-3 files first. Refine the prompt on small batch failures, then run
the full set. --allowedTools only skips the permission prompt for the tools it names — it does not
bound which tools the session may call (the CLI reference says so outright: "To restrict
which tools are available, use --tools instead"). Under --permission-mode auto a
background classifier, not that list, decides what else may run, so this recipe alone does
not keep the fan-out inside Read,Edit,Bash(git commit *). For a fan-out that must not
stop for input and must stay close to an exact list, use --permission-mode dontAsk: it
auto-denies anything that would otherwise prompt, running only actions matching your
permissions.allow rules, read-only Bash commands, and calls a PreToolUse hook approves.
Scope-Verified Parallelism¶
When multiple sessions fan out and write to the same codebase, file collisions cause corrupt merges and lost work. Prevent this at the hook layer rather than through manual file-locking discipline:
// .claude/settings.json
{
"hooks": {
"PreToolUse": [
{
"matcher": "Edit",
"hooks": [
{
"type": "command",
"command": "scripts/check-file-lock.sh"
}
]
}
]
}
}
The hook command is a write-claim check: before an Edit lands it asks whether another live
session already owns that file, and exits 2 (deny) if so. In
Kanevry/session-orchestrator that claim is
held by a JS lock-module family (scripts/lib/session-lock.mjs,
scripts/lib/locks/state-md-lock.mjs) which grants ownership only after verifying a second
identity factor — pid, host and start time — because a semantic session label is not globally
unique. (That repo contains no check-file-lock.sh, and its STATE.md schema has no
acting_on field; treat the shell name above as a placeholder for your own hook script.)
This enables safe parallel fan-out without coordination overhead: each agent checks at the point of write, not at the point of task assignment.
(session-orchestrator — Kanevry/session-orchestrator, Jun 2026.)
A complementary check earlier in the pipeline: rather than only catching a collision at the point of write, a classifier-gated pre-claim check can mark candidate parallel work green/yellow/red/blocked before dispatch, based on file overlap, shared dependencies, and shared infrastructure — reducing how often the write-time lock above actually has to deny anything. See Task-Shaped DAG Orchestration for the full pattern; the two checks are complementary layers, not substitutes for each other.
A remote lock for coordination that must survive across clones and worktrees, not just
within one working copy. Kanevry's write-claim above protects agents sharing one working
copy; a git-ref-based lock extends the same idea to agents in separate worktrees or
clones by publishing the claim as a ref on the remote: refs/loop-lock/<feature>.<epoch>.
Race resolution is deterministic without a coordinator: "after publishing, the acquirer
re-lists the lock refs and keeps the lock only if its own ref sorts first — two racing
acquirers resolve deterministically, the loser backs off." A lock older than a stale-hours
threshold (2h by default) is treated as abandoned and reclaimable. It degrades gracefully
when there is no shared remote to publish to — falling back to a local-only lock file
rather than failing closed.
(muzafferkadir/nextjs-loop-engineering-starter,
Sep 2026.)
Multi-Loop Coordination¶
For repos running multiple distinct loops (CI Sweeper + PR Babysitter + Dependency Sweeper, etc.), fan-out coordination extends to cross-loop ownership:
- Each action loop writes
acting_on: <branch-or-pr-id>in its STATE.md before starting any work on that item - All loops read all STATE.md files before claiming a new item — skip items already claimed by any other loop
- Priority ordering determines who acts first when two loops find the same work
- Cross-loop conflicts that cannot be auto-resolved go to a human inbox in STATE.md
See Loop Patterns for the full priority ordering and eight named loop patterns, most with defined STATE.md coordination (Thin Loop keeps none).
Agentic MapReduce¶
For codebase-wide analysis tasks (security audits, dependency sweeps, dead-code detection), structure the fan-out as an explicit MapReduce rather than an unstructured swarm:
- Map — a coordinator agent scans the codebase and maps out signal locations (files, patterns, or regions likely to contain what you're looking for)
- Fan out — each mapped region is assigned to its own bounded agent invocation, scoped to only that region (same scope-verified parallelism as above)
- Reduce — a synthesis pass collects all per-region findings into a single deduplicated list
- Verify — each candidate finding is independently confirmed in an isolated sandbox before being reported, so the reduce step doesn't just aggregate unverified guesses
The map stage is what separates this from plain fan-out: instead of dividing work mechanically (one agent per file), a signal-aware pass decides where to look first, so bounded-context agents spend their budget on high-likelihood regions rather than scanning uniformly.
(Devin, "Security Swarm", Jul 2026.)