Long-Running Agents¶
A long-running agent maintains progress across hours or days, spanning multiple context windows. Unlike a single-session agent, it must be designed like a distributed system: assume the process will be interrupted and build recovery in from the start.
The Core Challenge¶
Each context window is finite. A long-running agent must externalise its state so the next invocation can resume exactly where it left off, without re-reading the entire history.
Planner–Worker–Judge¶
A three-role multi-agent pattern for complex, multi-stage tasks:
- Planner: breaks the overall goal into concrete, bounded sub-tasks; writes a task queue to a persistent file
- Worker: executes the current sub-task; commits the result; marks the task done in the queue
- Judge: evaluates output; either approves (advance to next task) or sends back with specific feedback (re-enter planner phase)
The Planner writes state; the Worker reads and updates it; the Judge is the loop's stopping condition.
(Wilson Lin, "Scaling long-running autonomous coding", Cursor, Jan 2026.)
A related but distinct single-agent technique: the Ralph Loop. Where the pattern above splits work across separate Planner/Worker/Judge roles, Geoff Huntley's "Ralph Wiggum" technique deliberately keeps it to a single monolithic agent process run autonomously in a loop, one task per iteration, prioritizing determinism and watchability over multi-agent complexity — the community precedent Osmani and others cite as predating Claude Code's own built-in loop primitives. (ghuntley.com, "Ralph Wiggum as a 'software engineer'", Jul 2025.)
A multi-day case study at scale. Iterative planning-coding-testing loops sustained over multiple days report a 52.25% average relative gain across three benchmarks and, as a demonstration, autonomously build a working FPS game over 70+ iterations — see Harness-of-Harness for the full result. The persistence discipline underneath a run that long is the "megathread" pattern — keep one agent thread alive and compact it on purpose rather than restarting fresh each task — see Harness Patterns for the codex-maxxing source.
Persistent State Management¶
Everything that must survive a context window reset must be written to a file or committed to git:
PROGRESS.md— current task, completed tasks, blockersPLAN.md— full task queue generated by the planner- Git commits — atomic unit of verified progress; the recovery point if the agent crashes mid-task
Recovery Pattern¶
At the start of every invocation, the agent reads its state file before anything else:
Before starting any work:
1. Read PROGRESS.md — understand what was completed and what comes next
2. If the last task is marked IN_PROGRESS, treat it as incomplete and restart it
3. If PROGRESS.md does not exist, this is the first run — create it now
This pattern turns any crash into a clean resume.
Inner/Outer Dual Loop¶
When a long-running agent insistently fails on a sub-task, the inner loop alone cannot escape the deadlock. The Dual Loop adds a strategic layer above the execution loop:
Outer loop (strategy):
→ sets the goal and approach
→ monitors inner loop health (success/failure counts)
Inner loop (execution):
→ attempts the current sub-task
→ reports success or failure with reason
If inner loop fails N consecutive times:
→ outer loop triggers a strategy reset
→ re-evaluates the approach, adjusts the plan
→ re-enters with a different strategy
The key insight: repeated failure is a signal about the strategy, not the execution. The outer loop intervenes before the inner loop burns its budget on a stuck approach.
This is a different axis from the Three Feedback Loops: the Dual Loop nests execution inside strategy (both can be automated); the three feedback loops nest agent inside developer inside user by who supplies the correcting signal and at what cadence.
The outer-loop/inner-loop split traces to Microsoft's Magentic-One multi-agent architecture (the Orchestrator's task ledger drives the outer loop, its progress ledger the inner loop) — see "Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks" (Microsoft Research, Nov 2024). (Framing here follows Data Science Dojo, "Agentic Loops: From ReAct to Loop Engineering", Jun 2026.)
Real-World Scale¶
/goal loops have demonstrated remarkable autonomous reach on a large, mature codebase. Sam Willis directed Codex agents through a multi-week /goal-driven rewrite of PostgreSQL for native threading: 1,000 commits · 124,000 lines changed · 786 files modified over 10 days — though Willis notes the diff includes planning docs and a work log that inflate the changed-line count by about 35%, and the run was not unattended: he wrote and refined the plan up front, then guided the branch through goals, review, and benchmark interpretation across many separate /goal sessions (one phase, Phase 12, ran unsupervised for up to four days). (samwillis/multithreaded-postgres README, Jun 2026.)
The limiting factor was not raw model capability but the quality of each phase's plan and the human review between /goal sessions.
A cross-domain reference point: 100% on ARC-AGI-3. NVIDIA's AVO agent completed the full 25-environment public ARC-AGI-3 set with a 100.00 RHAE score, solving all 183 levels in 6,624 environment actions — using roughly 12% fewer actions than the prior best (not a controlled ablation; the comparison agent differs in memory and context management too). The architecture is the same hypothesize → act → observe → update-state → continue loop this doc already documents, built on two mechanisms: persistent memory carrying forward implementations, eval results, and profiler output across the run, and a supervisor that monitors the broader trajectory for stagnation and can redirect the main agent — a concrete instance of the Inner/Outer Dual Loop above, at a different scale. As a demonstration of the same architecture applied outside a benchmark, AVO ran continuously for 7 days, explored 500+ optimization directions, and produced 40 committed GPU-kernel versions that outperformed cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5%. (NVIDIA Developer Blog, "NVIDIA AVO reaches 100% on ARC-AGI-3", Aug 2026 — not independently re-verified beyond the vendor's own post; treat the specific numbers as NVIDIA's self-reported figures.)
An 11-day, dozens-of-agents case study: formalizing Fermat's Last Theorem. Dozens of Claude agents ran in parallel over 11 days (~6 billion output tokens) to produce a 13-million-line Lean formalization of Wiles's proof, verifying 30,300 theorems along the way (29,500 appearing in the final proof). The coordination mechanism that made this tractable — credited by the reporting as fixing an earlier, unscaffolded multi-agent attempt that failed to coordinate — was Prove2Me, a third-party open-source platform maintaining a DAG of theorem statements and dependencies so parallel agents don't duplicate work or lose track of state, with statements and proofs kept in separate files (faster Lean recompilation) and natural-language node descriptions for semantic search. This is a concrete instance of externalized, shared state as the coordination substrate for many long-running agents at once, the same principle Fleet Engineering documents for coding fleets, applied to a proof-formalization fleet instead. Caveats worth keeping attached to this result: it formalizes Wiles's 1995 proof rather than producing new mathematics, builds on a large body of prior community formalization work (106 upstream files credited), and this is third-party reporting on an Anthropic research post/repo, not a formal Anthropic press release. (Tech Times, reporting on Anthropic's Fermat's Last Theorem formalization, Sep 2026.)
A second many-agent long-horizon research harness, this time for mathematics/theoretical CS rather than proof formalization. A model-agnostic system using readiness gates and tree-combination aggregation (rather than Prove2Me's DAG-of-dependencies) reaches 71.0% on TCS-Bench. (arXiv 2609.15983, "Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical CS", Sep 2026.)
A third: an 8-day run where the agent improves its own research loop, not just the research output. Rather than executing a fixed research pipeline for days (Stellar Colosseum, AVO), this system recursively rewrites its own control loop over an 8-day unattended run, discovering 7 successive enhancements (new search policies, new memory mechanisms) that generalize to 4 held-out benchmarks and — notably — reduce reward hacking rather than introducing more of it, a favorable direction given how often self-modifying loops trend the other way (see Common Failure Patterns' reward hacking entry). This is the long-horizon-agent question from a different angle than the rest of this section: not "how far can one agent get unattended" but "can the loop itself keep improving across that same unattended run without a human retuning it." (arXiv 2609.26457, "AIDE²", Sep 2026.)
A cross-vendor, computer-use example of the same unattended-persistence pattern applied to creative-tool orchestration rather than a code-only environment: ChatGPT Astra reportedly coordinated local desktop tools (Windows, Unreal Engine 5, Blender) unattended overnight to build a game trailer. (@Sabrina_Ramonov, Sep 2026 — single-source practitioner report, unconfirmed by OpenAI.)
A first result from Anthropic's own life-sciences lab, with a published skeptical read attached. Anthropic set 949 Claude agent sessions — a single launch session plus 414 worker, 375 supervisor, 107 curator, and 52 editor sessions, running on Claude Mythos 5 — to survey reverse-transcriptase loci across 1.9 billion metagenomic protein clusters, with only high-level direction from the team's scientists. The campaign ran 119 tasks over 21.5 hours of wall-clock time, "without human intervention," using 77 agent-hours and 215.6 million tokens (11.3M uncached input, 14.9M output, 189.5M written to the prompt cache). Reading raw DNA sequence directly into its context, one worker agent wrote: "I can see by eye a tandem repeat array … that's a CRISPR-like or msDNA-like repeat array?!" — then confirmed in its own literature search that the pattern matched no system already described in the literature before filing a report. What it had flagged, since named array-associated reverse transcriptase (ART), "consists of three parts: the RT, a partner gene beside it, and a long array of evenly spaced DNA repeat sequences" (the array itself is non-coding) — an architecture that resembles a CRISPR array without a demonstrated CRISPR-like function. On process: each candidate report only "proposes a function"; when one survives review and reaches the lab, human scientists design and run the confirmatory experiments, with Claude "helping to interpret" the resulting data. Feng Zhang, a CRISPR pioneer at MIT and the Broad Institute, called the finding "genuinely intriguing" after reviewing the preprint. Nature News supplies the outside caveat the campaign's own framing omits: "there is scant evidence that the repeats found by Anthropic's team of AI agents have similar functions, and there is no known DNA-slicing enzyme partnered with them." (Anthropic, "Claude discovers a novel enzyme system with CRISPR-like repeats", Sep 2026; Yoon et al., "Autonomous AI agents discover reverse transcriptases with tandem repeat arrays" (preprint), Sep 2026; Heidi Ledford, "Anthropic's AI biolab finds 'CRISPR-like' DNA in viruses. What's next?", Nature News, Sep 2026.)
A test-pruning loop still running two weeks on (the maintainer's own counters). OpenClaw's maintainer posted on 24 Sep 2026 that the project
"deleted around 400k LOC of its own tests without much change in code coverage" (post; see also
Verification). The loop kept going: "That /goal is still running." on 29 Sep (post),
"So far deleted 600k of the slop part" on 30 Sep (post), and "Still running that loop, now at about 1.3Mio deleted weak
tests, CI times twice as fast." on 8 Oct (post). Treat these as counters, not measurements: the unit moves between posts
(LOC, unstated, "tests"), none comes with a coverage measurement or CI timing beyond the words quoted above, and the posts do not say which tool's /goal runs it. The repo's own skill pulls the other way. On main
(read 2026-10-19; SKILL.md last changed 2026-10-12, CAMPAIGN.md 2026-10-13), test-audit/SKILL.md says audits "never chase deletion counts"
and states no numeric target, and its CAMPAIGN.md bounds a whole-subsystem prune as eight steps where
"Each step ends on its completion criterion; do not start the next step early": a pass/fail baseline pinned to a main SHA (in the Telegram campaign the baseline failures were "real delivery bugs, not stale
tests"), lanes along production-owner boundaries, a per-test ledger by read-only agents, a layer plan, cutover, an independent preservation review (the Telegram one "found nine real
gaps and one unreachable assertion") with a deliberate mutation per restored contract, a product-defect step, and a reconcile step because "Campaigns outlive many main commits". What the
posts report is a running count; the repo bounds the work by per-step criteria plus a review that tries to put back what was deleted. Whether the /goal run follows CAMPAIGN.md is not stated.
Detaching from the terminal¶
Long-running agents often need to run after you close your IDE. Two shapes, illustrated with Claude Code — most agent CLIs offer an equivalent detach, and most vendors now offer a hosted path:
claude --bg— starts a background session on the same machine; survives terminal close; manageable viaclaude agents. See Background Agents.- Routines — Anthropic-hosted cloud execution; survives machine shutdown entirely. See Routines.
Session Watchdog and Hard Time Limits¶
A long-running agent that never stops is not a feature — it is a bug waiting to become an incident. Three complementary limits:
| Mechanism | Bounds what | Implemented by |
|---|---|---|
| Turn cap per invocation | Bounds a runaway loop | A harness flag, in-process (--max-turns in Claude Code) |
| Cost cap per invocation | Bounds spend | A harness flag, in-process (--max-budget-usd in Claude Code) |
| Session watchdog | Wall-clock time; crash recovery | An OS supervisor, outside the process — harness-independent by design |
The watchdog is the layer the other two cannot cover: it detects stalled processes, enforces an absolute wall-clock limit (e.g. 2 hours), and restarts crashed sessions cleanly from the state file without human intervention.
Implementation pattern (systemd or equivalent — not a shell loop, which can itself become a zombie; the supervisor has to be able to outlive the thing it supervises): 1. Agent runs until it completes or hits the 2-hour limit 2. On limit hit, the agent sleeps and externalises its current state 3. Watchdog detects the sleep, waits the configured interval, and re-invokes 4. On crash (non-zero exit), watchdog immediately re-invokes from state file
This prevents zombie agents — processes that are running but producing nothing, neither failing nor succeeding, consuming tokens and budget indefinitely.
(clem — jahwag/clem, Jun 2026.)
A narrower, in-process cousin. The watchdog above handles a stalled or crashed process.
Claude Code also aborts a single hung MCP tool call from inside a live session: before
v2.1.187, a remote MCP tool call that received no response could block a turn indefinitely —
the same zombie failure, at the scale of one tool call. It now aborts after an idle timeout, set
by CLAUDE_CODE_MCP_TOOL_IDLE_TIMEOUT (configurable, or 0 to disable): 300000ms (5 min) by
default for network MCP servers (HTTP/SSE/WebSocket/claude.ai connector) and 1800000ms (30 min)
for stdio servers. For the API-retry watchdog (CLAUDE_CODE_RETRY_WATCHDOG,
CLAUDE_CODE_MAX_RETRIES), see Headless Mode → Fault-tolerant headless
loop.
(env-vars reference; CHANGELOG.md, v2.1.187.)
Control-Plane Continuity: LoopX and deep-loop (Oct 2026)¶
Two multi-session orchestrators show what "read the state file first" becomes once the state
is a database or a directory of runs rather than one PROGRESS.md.
- LoopX carries acceptance, evidence and the next action across iterations. Per the
release notes, v1.3.0 (2026-10-06)
came out of long-horizon benchmark diagnosis: a bounded replan context that retains
outcome and route diversity, Next Action writeback bound to the selected Todo, settlement
that follows the original authority, and checkpoint recovery.
v1.4.0 (2026-10-11, flagged
as a breaking change) makes canonical SQLite state with
hard_leasethe default for new unconfigured Goals, adds an implicit review cadence of six settled work Turns, evidence-backed work-Turn review and reviewed SQLite recovery. v1.4.2 (2026-10-15) makes recovery retain the original operation identity; v1.4.1 adds missing-task-context and stale current-basis checks. Read together (this KB's reading, not the release notes' wording): recovery restores the original authority and identity of an operation instead of re-deriving them, and review is triggered by a count of settled work, not by the agent's own sense that it is done. - LoopX's own long-horizon benchmark, with its caveats. The README reports a same-model, three-mode comparison on Long-Horizon Terminal-Bench (46 tasks): LoopX 1.0.3 Heartbeat 0.4948 mean reward, Plain Codex 0.4218, native Codex Goal 0.4475 (self-reported). It also caveats that some Heartbeat replacements ran with unequal runtime budgets and that strict pass counts do not improve over Plain Codex (7/46 each); the study date is not stated on the page. A usable template for reporting a loop harness honestly: mean reward and strict pass counts side by side, budget inequality disclosed.
- Long-lived loop state needs bounded lookups. deep-loop v1.27.0 (2026-10-07) documents a failure of the recovery path itself: with more than 64 run directories, or a stale absolute worktree claim, the PreCompact and SessionStart hooks failed for every session, so no compact checkpoint was written. The fix adds explicit bounds (256 runs, 64 full verifications, time caps) and distinguishes worktree claims of finished runs from active ones. Accumulated run history is a scaling input to resume, not just disk usage.
- Observability without write access. deep-loop v1.26.0
(2026-10-06) adds a read-only
run statusJSON envelope (budget, comprehension debt, breaker, pending human reviews, workstreams, next action) and a Claude Code status band with toasts for breaker, debt-gate, budget-stop and completion events; the UI reads the control plane but cannot change it.
For the per-turn admission side of LoopX see Loop Contract → Quota-Aware Should-Run Gate.
Relationship to Cost Control¶
Long-running agents are the highest-risk scenario for cost runaway. Always pair with:
- --max-budget-usd on every invocation
- Explicit STOP conditions in the PLAN.md
- A human checkpoint after each major phase (see Human-in-the-Loop)