Retrospectives

ℹ️ Version numbers below reflect the convention in use when this was written. Unbuilt work is now named, not numbered (STATUS.md). This record is left as it was.

One section per increment of the project's build plans (plans/), written during that increment's retrospective review (the workflow is in CLAUDE.md). Only findings worth keeping land here: a real defect the review caught, or a fact that would be expensive to rediscover. Fixes themselves live in the commits; this file records what was learned.

Every heading and claim here carries YYYYMMDD HH:MM (local, 24h) — several increments can land in one day, and a bare date loses their order.

Severity follows the design review's scale: HIGH — wrong behaviour or false confidence; MEDIUM — would block or mislead; LOW — worth remembering, not urgent.

The seven pre-implementation design review passes are at the foot of this file: Design review passes 1–7.

Start here — by what you are about to touch

Added 20260801 01:11. Forty-odd sections in date order is an archive, not something anyone reads before starting work — so this table is the way in. It is keyed on what you are about to do, not on when the lesson was learned, and it is deliberately short: only the classes that have recurred. A rule that hardened into a standing instruction lives in CLAUDE.md or BUILDING.md; this file is where the evidence for it is.

About to…	Read	Because
write a test	I2, L3–L4, L5, L5b	The recurring defect of this whole project: an assertion satisfied by something other than the property it names. A test that could not fail; a field with no assertion is a field that can be a constant; a tidy fixture defeats a mutation test; test the discriminating case, not the two sides separately; a fixture can be right for the wrong reason and hide the defect it was written for
claim a test is mutation-verified	L5b, L3–L4	"Mutation-verified" is a per-assertion claim, never a per-commit one. A failing test proves the mutant is caught, never that it is caught for the stated reason — under -x, or when the failure lands on an earlier assertion, the one encoding the claim never runs
touch a sidecar or any YAML	I5, L5b	Writes must be rename-atomic. An explicit empty value was silently deleted on round-trip. Swapping a YAML library is not a swap; existing[:] = keep wipes ruamel's comment metadata outright; a comment before a sequence entry belongs to the entry above it; a warning is not an error, and a library that downgrades one is changing behaviour
add a gate or a check	I7a, L3–L4, the eval-harness section	A gate that has never been shown to fail is a claim, not a check. A gate that never reads the artifact it guards is checking a copy. The free-path gate was defeated by its own harness. The exit criterion was the thing nobody ran
write an error message	L3–L4, I8	An error message is part of the interface; a remedy inside one is a claim, and it was false. A fix applied to one surface is half a fix
change retrieval	I9, the eval-harness section, I6	Three defects under one green suite. Overlap could push a chunk past max_tokens; heading text landed in no chunk at all
edit a doc, a plan or an exclusion list	L5b, I9, the shared-file section	An exclusion list is a set of claims, and claims rot. A fix instruction can carry its own defects. Four silent str.replace no-ops in one session — an edit that does not match is an edit that did not happen. A clean auto-merge is not a correct merge
cut a release	the post-v0.1 housekeeping section, the two release-sweep sections (20260811)	A CHANGELOG entry and a __version__ are only claims: 0.1.0 had both for two days with no tag, no release and nothing published. Verify with git tag -l, gh release list, and the index itself. A release sweep is table-shaped and misses prose — grep for the superseded version number — and ordering is a property of the sequence: read the sequence, not the neighbourhood
trust CI	the cross-platform scanned-fixture section	main was CI-red for three pushes and nobody noticed. Green proves the tests ran, never that they can detect the defect
run a mutation pass	G1, L6 § Mutation testing, T3, T7, E5	Commit before mutating — git checkout restores to the last commit and has silently reverted uncommitted fixes six times. A harness must prove it can kill before its silence means anything: assert the anchor matched exactly once, clear __pycache__ between mutants, run without -x, restore from a copy. A killed run poisons everything after it, and a surviving mutant first asks is this reachable at all?
write a measurement tool, or publish a number	the reachable-ceiling probe, measure_sync_cpu.py, E2, E6	A tool that absorbs input it cannot measure reports a plausible number — strictly worse than a crash. Validate everything the measurement consumes, not the bugs already reported; the artifact names every input its numbers are a function of; one test runs the tool the way it is really invoked, because the shipped one measured the launcher and not the work. A probe is code and earns the same adversarial pass — two of five constants were measured wrong on E6's first run
lean on a fake, a seam or a fixture	E6, I7b (passes 6–7), the [light] backend error, Running it found what reading it could not	A seam built for testability defines a region no test reaches, and that region needs its own gate: the deep client's schemas 400d on every live call for four releases of green fixtures. "Each part works" is not "the parts are connected". For any new knob, turn it on a real KB before landing — only running proves the parameter arrives
apply a review fix	T3, T4, L6, 2d round two	A fix applied under review inherits the review's confidence and none of its scrutiny — every review loop here has found its worst defects inside the previous pass's fixes. "Pinned by test X" is a claim about a failing test: revert the fix and watch it go red, or do not write the word. Re-run the whole battery after every fix — a fix can silently disarm a test written for something else

The pattern across all of them, and the reason this file is worth keeping: the worst finding of a review pass is usually inside the previous pass's fix. That has held from I5 through L6's thirteen rounds, and it is why a fix is re-reviewed rather than assumed.

I1 — Package skeleton, errors, CLI dispatch (20260725 13:40)

MEDIUM — PinakesError could not be pickled, so an error crossing a process boundary raised TypeError instead of reporting itself. Exception.__reduce__ replays self.args through type(self), but every subclass here takes its own constructor arguments (NotImplementedYetError takes a command name and an increment), so rebuilding blew up on the missing remedy keyword. Confirmed by probe before fixing, not reasoned about. Fixed with an explicit __reduce__ routing through a module-level helper. Lesson: an exception class with a non-(message,) constructor is unpicklable by default — the failure only surfaces under xdist/multiprocessing, i.e. exactly when something else has already gone wrong.

MEDIUM — the subcommand dispatch target sat on the public namespace attribute run. Any future command declaring --run would have silently overwritten the function main() then calls. Moved to a reserved _runner dest with a test asserting no public namespace attribute is ever callable. Lesson: set_defaults shares one namespace with every option; anything the framework itself dispatches on must be underscore-reserved.

LOW (reference) — measured ty 0.0.63 against pyright strict on a 6-defect probe. pyright caught 6/6; ty caught 1/6 (the str | None → len error, with better diagnostics); ruff caught the unused import. ty currently has no strict mode: it accepts unannotated defs and Any leakage, which is precisely what pyright strict is in this project for. Decision (user): keep pyright as the gate, add uv run ty check as a fast pre-check. Worth re-measuring when ty leaves beta — the gap is a missing feature, not a design difference.

I2 — ULID identity and pnk:// URIs (20260725 14:05)

MEDIUM — a test that could not fail. test_an_unresolved_uri_cannot_be_formatted asserted not hasattr(parsed, "__str__") or "pnk://" not in str(parsed). Every object has __str__, so the first clause is always false and the second only checked a dataclass repr — the test would have passed even if ParsedUri had grown a full URI renderer. Replaced with a precise structural assertion ("__str__" not in ParsedUri.__dict__, present on PnkUri) that names the static guarantee as primary. Lesson: a green test asserting a tautology is worse than no test — it buys false confidence, and hasattr(x, "__dunder__") is almost always one.

MEDIUM — a docstring claimed more than had been verified. ids.py said python-ulid rejects the ambiguous Crockford letters I, L, O, U; only I and U had actually been probed. All four are now probed and the claim is stamped with the time of that probe. Lesson: when writing "verified X", the set being claimed must be the set that was run — a partially-probed claim reads identically to a fully-probed one.

MEDIUM — two except Exception blocks wrapped calls whose only expected failure was InvalidIdError, so a TypeError from a future refactor would have been re-raised as "this is not a valid KB ULID". Narrowed, and ruff's BLE ruleset enabled so the class of mistake cannot be written again.

LOW — internal helpers were public. parse_kb_id_for_uri/parse_doc_id_for_uri took an odd (raw, segment) pair and had no business in the module's API; renamed to _kb_segment/_doc_segment.

LOW — the scheme is matched case-sensitively while the self sentinel is not. Deliberate (URIs are machine-written, self is hand-typed) but undocumented; now stated in the module docstring and covered by a test.

I3 — Manifest parsing and KB root discovery (20260725 14:25)

MEDIUM — an explicit empty value silently became the default. timezone = "" in [budget] read back as "UTC", because the accessor ended in or "UTC". Same shape accepted name = "" and model = "" outright. All three confirmed by probe. Empty strings are now rejected with a named key, and the default only applies when the key is absent. Lesson: value or default conflates "missing" with "empty", and for user configuration those mean opposite things — one is silence, the other is a mistake worth reporting.

MEDIUM — narrowing by assert. Two call sites used assert value is not None to convince the type checker, and a _require fallback built a Path out of a table name for an error that could never fire. Python strips asserts under -O, so the "guarantee" was a comment with syntax. Replaced with three explicit accessors — string (required, returns str), optional_string, string_or — which give the type checker what it needs without a runtime claim. Lesson: when a type checker needs an assertion, the API shape is usually wrong; fixing the signature beats asserting.

MEDIUM (docs) — the required/optional split existed only in code. [chunking], [retrieval], [rerank] and [budget] are optional with documented defaults while [kb], [sources] and [embedding] are mandatory — a user-facing contract that docs/DESIGN.md §2.1 never stated. Added there in the same change, per the repo's docs rule.

LOW — a test caught a real error-message defect. Validation errors read [<root>.retrieval]; the table path is meant to name what the user would type. The failing test was fixed in the source, not the assertion.

I4 — SQLite storage, FTS5 triggers, vector loading (20260725 14:45)

MEDIUM — load_vectors peaked at roughly twice the array it returned. It collected every embedding into a list and vstacked it: measured 669 MB for 200k×384, where the result is 307 MB. At 1M chunks that is ~3.4 GB against the ~1.5 GB §3.1 states, so the design's own memory claim was wrong for its only shipping tier. Now counts first and fills a preallocated array, with a test asserting peak < 1.6× the result. Lesson: "load it all into one contiguous array" has two implementations that differ by a factor of two, and only one of them matches what the design promises.

MEDIUM — a non-database file produced a raw sqlite3.DatabaseError. PRAGMA journal_mode is the first statement to touch the file, so it failed before any schema check could run, and the user got file is not a database with no remedy. Opening is now wrapped. Lesson: the friendly check ran second; the pragma that configures the connection is what actually opens the file.

MEDIUM — chunk insertion restarted ordinals at 0 on every call, so a second call for the same document hit the UNIQUE (doc_id, ordinal) constraint. The operation is really a wholesale replace — re-chunking must not leave old chunks searchable — so it now deletes first and is named for that. Both this and the one above were caught by tests written in the same increment.

MEDIUM — pickling collapsed every error subclass to PinakesError. I1's __reduce__ fix rebuilt through the base class, so StoreError came back as PinakesError and any except StoreError on the far side would miss it. Now rebuilds the original class via __new__, keeping message and remedy. Lesson: a fix that makes an object survive a round trip is not the same as one that makes it survive intact — check identity, not just contents.

LOW — DOCUMENT_STATES/LINK_ORIGINS duplicated the DDL's CHECK constraints with nothing tying them together. A test now fails if they drift.

I5 — Sidecars (20260725 14:55)

HIGH — sidecar writes were not atomic. path.write_text truncates before it writes, so a crash or full disk mid-write leaves a truncated sidecar — and the one thing a sidecar carries that cannot be recomputed is the document's permanent ULID. Losing it breaks every inbound pnk:// link, and no later command can repair it, because nothing else knows what the id was. Now writes to a sibling temporary and os.replaces over the target. Lesson: "the truth is in files" makes every file write a durability question; the ones holding unrecoverable identity deserve rename-based atomicity.

MEDIUM — an explicit empty value was silently deleted on round-trip. tags: [] and provenance: {} vanished, because write tested truthiness. This is exactly the lesson I3 recorded one increment earlier — absent and empty are different statements — and I repeated it in a module whose entire contract is "do not lose what the user wrote". The Sidecar now records which known keys the file carried. Lesson: a recorded lesson only helps if it is re-read while writing the next thing that could break the same way; the pattern to watch for is any if value: guarding output.

MEDIUM — a hedged test assertion. assert made.title == "my research notes.md" or made.title == "my research notes" accepted either answer because I had not checked which one Path.stem gives. Two plausible values means the test asserts nothing about the one that is correct. Pinned to the real value. Lesson: an or in an equality assertion is the tautology smell from I2 wearing a different hat.

LOW — KNOWN_KEYS could drift from what write emits, which would be silent data loss for a key the module claims to understand. A round-trip test now asserts every known key comes back.

I6 — Structural chunking (20260725 15:10)

HIGH — overlap could push a chunk past max_tokens. The carried-over tail was prepended unconditionally, so overlap = 9 with max_tokens = 10 produced 12-token chunks (probed). Those are exactly the chunks the model truncates at encode time, silently — the failure §4.6 exists to prevent, reintroduced by the feature meant to improve context. The carry is now dropped when it would breach the limit, and a parametrised test sweeps the whole (max_tokens, overlap) matrix. Lesson: the earlier test used overlap=5, max_tokens=15 and passed; one comfortable ratio proves nothing about the boundary. Sweep the matrix when two parameters interact.

HIGH (design, found by a test) — heading text landed in no chunk at all. Headings were consumed as pure structure, so a word appearing only in a heading was unsearchable: the FTS index sees chunk text and nothing else. heading_path looked like it covered this, but it is a separate column that v0.1 never searches. Headings are now part of the first chunk beneath them, and docs/DESIGN.md §4.6 says so. Lesson: "the information is still recorded somewhere" is not the same as "the information is still retrievable"; check which column the query path actually reads.

MEDIUM — sentence splitting silently gave up on text without punctuation. A long run with no .!?;: produced one oversize piece that was emitted whole. Now falls back to words, then to characters for a genuinely unbroken run (a hash, a base64 blob).

LOW — piece offsets came from a running total that finditer's empty end-of-string match could desynchronise from the source. Now taken from match.start(), so spans are exact by construction rather than by accounting.

LOW (test-design) — a stand-in counter can be wrong in the direction that hides the bug. The word-counter says a 400-character unbroken run is one token; every real tokenizer disagrees. The character-cut path needed a token-dense counter to be exercised at all.

I7 — Embedding backends and reranker (20260725 15:35)

MEDIUM — a fake that could never disagree. The test backend was registered as FakeBackend(section.dim), so it reported whatever width the manifest claimed — making the dim-mismatch check, the one guard against storing incomparable vectors, impossible to test. Pinned to a fixed width. Lesson: second time in three increments that a stand-in was wrong in exactly the direction that hides the bug (I6's word-counter was the first). A fake that derives its answer from the input under test asserts nothing.

MEDIUM — an assertion guessed at a real model's behaviour. count_tokens("retrieval augmented generation") > 3 was written before any model had been run; the real BPE count is exactly 3, and the test failed the moment weights were cached. Rewritten to assert the relationship (longer text → more tokens) rather than a number I had invented. Lesson: when a test crosses into a real dependency, assert properties, not remembered values.

Verified against real weights (not inferred): fastembed's BAAI/bge-small-en-v1.5 gives dim=384, max_seq_length=512 derived from the tokenizer's truncation config, normalised float32 vectors (self-cosine 1.0), and it wrote to ~/.cache/huggingface/hub rather than $TMPDIR/fastembed_cache. BAAI/bge-reranker-base ranked a relevant passage above an irrelevant one, -0.28 vs -7.85.

LOW (reference, matters for I14) — reranker scores are raw logits, not probabilities. They came back around -0.28 and -7.85, not in [0, 1]. The illustrative thresholds in §2.1 (low_below = 0.31, high_above = 0.62) read like normalised scores; calibration must either fit against the logit scale or squash it first. Recorded now so I14 does not quietly fit thresholds to the wrong scale.

I8a — Sync pairing core (20260725 15:50)

MEDIUM — a model-test guard checked the wrong half (found in I7, surfaced here). The model-marked tests skipped when weights were absent, but not when the backend was absent. Model weights live in a shared machine-wide cache, so a worktree without the light extra installed still saw them, ran the test, and failed with BackendMissingError instead of skipping. Only noticed because a second worktree had a different install set. Now both halves are checked. Lesson: a skip condition is a claim about the environment, and machine-wide state (a shared cache) is not evidence about the local one.

Design note, not a defect — DuplicateIdsError raises rather than returning an action. plans/20260725_1317-v0.1.md lists it among pair()'s return values. Raising is better: the condition is fatal for the whole run, and an action every caller must remember to inspect is one a caller will eventually forget. Recorded because it is a deliberate divergence from the reviewed plan.

What the exhaustive table bought. Writing one test per §6.4 row, then the compound cases the table cannot express, is what forced the two decisions the design left implicit: a sidecar disagreeing with the index wins (docs/ is truth, the index is derived), and a whole-picture rule must be order-independent — asserted directly by pairing the same snapshot walked forwards and backwards.

I8b — pnk sync, locking, and the rebuild swap (20260725 16:20)

HIGH — the rebuild swap left the old database's -wal/-shm behind. Design pass 2 fixed the missing checkpoint; this is the other half nobody had noticed. SQLite names the companions after the path, not the inode, so after os.replace they sit beside the new index claiming to be its write-ahead log — the exact corruption the checkpoint was added to prevent, reintroduced by the rename that followed it. They are now removed after the swap. Lesson: an atomic rename is atomic for one file; a WAL database is three files with correlated names, and correctness arguments about "the file" quietly skip the other two.

MEDIUM — a read-only SQLite connection creates -wal and -shm itself. This masked the bug above: the test read the index before asserting the companions were gone, and the read created them. Worth knowing beyond this test — §4.7 says the MCP server opens the index read-only "so it cannot write", which remains true of the data, but the server does create files in .pinakes/. Any future check that treats "no companions present" as evidence of a clean shutdown is wrong.

MEDIUM — a leaked connection inside a test helper. The helper was a generator, and a caller using next() left it suspended, so its finally: close() never ran. The symptom appeared in an unrelated assertion about rebuild. Now returns a list and closes immediately. Lesson: a generator with cleanup in finally only cleans up if it is exhausted; for a fixture-shaped helper, return the list.

LOW — ty caught a loose test shim pyright had been told to ignore. The monkeypatched __import__ took *args: object; pyright was silenced with an inline ignore, ty flagged the real mismatch. Typing the shim to __import__'s actual signature satisfied both and deleted the suppression. First time the "fast pre-check" found something the gate had been told to skip — noted, since I1's decision assumed ty would only ever be faster, not different.

I9 — Retrieval pipeline (20260725 16:55)

MEDIUM — vector search padded its candidate list with zero-similarity passages. argsort returns every row, so a query sharing no direction at all with a passage still ranked it, and with nothing better available those passages reached the user. Real models rarely produce an exact zero, so this would have hidden until a sparse or domain-shifted corpus hit it. Non-positive cosines are now dropped. Lesson: "return the top N by similarity" is only sane while similarity means something; N is a cap, not a quota to fill.

MEDIUM (design gap, decided here) — the design said the filter set included "date", and no date column existed. Documents carry mtime; a sidecar's created is optional, and filtering on an optional field silently excludes every document that lacks it — worse than having no filter. The filter is now mtime, and §4.1 says so. Lesson: when the design names a filter dimension, check which column actually holds it for every row, not just the well-formed ones.

LOW — sqlite3.Row hands back Any, which erased types through the whole hydration path. Rows are now narrowed once into a small frozen dataclass instead of being cast field by field at each use — pyright strict was the thing that made this visible.

Worth recording: FTS5 escaping is not optional. AND, OR, NEAR, *, " and a bare apostrophe are all parser syntax; a user typing it's would otherwise crash the query. Quoting each word as a literal and joining with OR keeps recall — an implicit AND drops a passage for one missing word, which is exactly the recall the vector half is there to provide.

I10 — pnk init, the notes template, pnk search (20260725 17:15)

MEDIUM — warnings were being printed and ignored. Turning filterwarnings = ["error"] on immediately produced two real problems that had been sitting in the summary: importlib.abc. Traversable is deprecated and removed in Python 3.14, so this project would have broken on the next Python it claims to support; and several tests leaked SQLite handles, surfacing as ResourceWarning raised from wherever the garbage collector happened to run — never the test that leaked. Lesson: a warning nobody has to act on is a warning nobody reads. Making them errors cost one afternoon of cleanup and bought a Python upgrade.

MEDIUM (process) — I committed with a failing gate. The last edit before committing introduced a reportUnusedFunction on an autouse fixture; I had run pyright before that edit, not after. Fixed in the retrospective commit. Lesson: the gate belongs immediately before git commit, not "recently" — and "recently" is exactly what it felt like at the time.

LOW — the test fixture violated a manifest invariant I had written myself. Setting max_tokens = 60 while leaving overlap = 64 was rejected by I3's cross-key validation. Pleasant confirmation that the check earns its place: the first thing it caught was its own author.

Design note — the template ships [retrieval.confidence] commented out. plans/20260725_1317-v0.1.md had already decided this, and building it made the reason concrete: pnk init cannot know anything about the corpus the user is about to add, so any threshold it wrote would be a number with no provenance. confidence: unknown until they fit their own is the only honest default.

I11 — pnk doctor (20260725 17:35)

MEDIUM — ty found a second real defect hiding behind a pyright: ignore. I had typed a dict as dict[Path, object] and silenced the resulting argument-type complaint rather than fixing the annotation. That is the same shape as I8b's finding, and it is now a pattern: an inline suppression is where a type error goes to be forgotten. Every pyright: ignore in src/ has been removed as of this increment; the two that existed were both hiding something real, not appeasing a checker that was wrong.

Worth stating — doctor is where several design promises stop being rhetorical. §3.1's linear scan ceiling, §6.2's link-coverage ceiling, §4.2's unknown confidence, §6.4's orphan reporting and §6.5's lock are all "we will tell you rather than pretend" commitments. None of them is honest unless something actually prints them, and until this increment none of them did. The test that says every non-OK check must carry a remedy is the one enforcing the spirit of it: a report that says "problem" without saying "do this" is just anxiety.

Design note — an uncalibrated KB is a WARN, not a FAIL. Reporting confidence: unknown is the honest behaviour the design chose, so a KB doing exactly that is not broken; it is uninformative, and the warning says so with a pointer to §4.2 and §7.

I12 — pnk install-hooks (20260725 17:50)

Confirmed end to end, with a real commit: the three-hook split does what design pass 6 claimed. docs/note.md and docs/note.md.pnk.yaml land in the same commit, and git status is clean afterwards. That was the whole argument for splitting the hooks, and it is now a test rather than a paragraph.

LOW — the pre-commit half needs no embedding backend at all, which only became obvious when a subprocess pnk ran without the test's fake registered and the sidecar half still worked. That is the right shape: minting an id is cheap and belongs before the commit; embedding is slow and belongs after it. Worth recording because it means a KB whose backend is not installed can still commit documents with correct, permanent ids — the failure is deferred to indexing, exactly where §4.5 says a core-only install should feel it.

LOW (test hygiene) — a "tree is clean" assertion failed on files the fixture never committed. The hooks were fine; the setup was. An assertion about cleanliness is only meaningful from a clean starting point.

HIGH (process) — I committed with red gates for the second time, and now understand why. The pattern uv run pyright 2>&1 | tail -1 && git commit … reports the exit status of tail, which is always 0. Both checkers were failing and the commit went through looking green. This is not carelessness that can be fixed by resolving to be careful: the shell was reporting success. Added check.sh, which runs every gate under set -e, and pointed CLAUDE.md at it. Lesson: if a safety check is a pipeline, the thing you are checking is the last command in the pipe. Make the gate a script that exits non-zero, and the mistake becomes unavailable.

I13 — pnk serve (MCP) (20260725 18:15)

The boundary is testable, so it is tested. §4.7's claim is that an agent cannot reach outside the KBs the server was pointed at. Three tests hold it: pinakes_get refuses a path, a traversal string and an unknown ULID identically; a KB that exists on disk but was not passed on the command line is unreachable and the error says arguments select by name or ULID never by path; and a document deleted since it was indexed cannot be fetched.

MEDIUM — stat()-based staleness detection works, and the test proves the thing the design argued about. After a --rebuild swap the server returns the new documents. The design's reasoning (an open handle pins the old inode, so meta.build_id read through it would report the old build forever) is now backed by a test that would fail if someone "simplified" it back.

LOW — pyright strict flags decorator-registered functions as unused. @mcp.tool() returns something pyright cannot tie back to the name, so all three tools looked dead. Rather than suppressing it, they are now registered in an explicit loop — which also makes the set of exposed tools one readable line instead of three annotations. A suppression would have been the third one this session that turned out to be hiding something; making the code say what it means was cheaper.

LOW — another leaked connection, caught by warnings-as-errors from I10. A Server built inline inside a pytest.raises block was never closed. This is the third leak that setting only found; before I10 it would have been invisible.

I14 — Demo KB, golden set, eval harness, calibration (20260725 19:00)

HIGH — the two error rates were a flattering zero, and it took looking at real numbers to see it. The first eval run reported false_abstain: 0.0 and false_confidence: 0.0. Both were vacuous: the demo KB had no fitted thresholds, so confidence was always unknown, so neither error could ever be counted. A CI gate on false-confidence would have passed forever, and passed loudest exactly when calibration was missing. Added confidence_coverage to the metrics, and made a drop in it a regression in its own right. Lesson: a perfect score on an error rate is a claim that deserves the same suspicion as a failing one — check the denominator before believing the ratio.

HIGH — the first threshold formula made low unreachable. low_below = min(answerable) came out at -9.885 on real logits, a floor almost nothing falls below, so the system could essentially never abstain and false-abstain was zero by construction. Both thresholds are now fitted from the unanswerable distribution — the only outcomes known absolutely — with low_below its median and high_above a high percentile. Only visible because the fit was run against real reranker scores rather than a fake's tidy 0-to-1 range.

The measured cost of the confidence heuristic, stated plainly: false-confidence is 0.25. One no-answer question in four still gets a confident answer, because the score distributions genuinely overlap (answerable -9.9..7.9, unanswerable -8.3..-2.7). §4.2 promised this would be measured rather than assumed; it is now in docs/DESIGN.md's risk table with its date and models. Two caveats are recorded with it: eight no-answer questions is a small sample, and the thresholds are fitted on the same set they are scored against, so it is a floor rather than a measurement.

MEDIUM — the test fixture copied .pinakes/ and ran the 64-dimensional fake against a 384-dimensional index. I4's stored-vector width check refused it, which is exactly right and briefly baffling. Generated state is now excluded from the copy. A good sign for the guard: the first thing it caught was a developer, not a user.

LOW — ruff caught assert ... or True in my own test. The tautology lesson from I2, third appearance, this time found by a linter rather than by reading. SIM222 earns its place.

I15 — CI, packaging, 0.1.0 (20260725 19:30)

MEDIUM — the version lived in two places. pyproject.toml and __init__.py each carried it, which is one place to forget on every release. Hatch now reads it from the module, and the release workflow refuses a tag that disagrees with it — a mismatched tag is the kind of thing nobody notices until an install pulls the wrong thing.

The wheel smoke test earned its place immediately. pnk init reads its template through importlib.resources, so a packaging mistake is invisible in the source tree and total after install. Running pnk init from the built wheel — and asserting the manifest, the golden-set stub and the .gitignore all appear — is the only check that would have caught it.

Publishing is left as a human step, on purpose. The release workflow runs on a v* tag and nothing else, and PyPI trusted publishing has to be configured in the PyPI UI first. Neither the tag nor the publish happens automatically from a merge: an irreversible, outward-facing action should need someone to mean it.

§8's v0.1 sentence, walked item by item: 17/17 present. The plan asked for that walk explicitly at this increment rather than trusting the accumulated sense that everything got done.

---

Post-v0.1 release housekeeping (20260727 15:35)

Not an increment — a session that merged the graph research, closed out the v0.1 plan, and cut the releases that had never been cut. Recorded because four of its findings are the kind that cost more to rediscover than to write down, and one is a mistake made in this session by the person writing this.

The 0.1.0 release existed in every artifact except the ones that matter. __version__ said 0.1.0, the CHANGELOG had a dated [0.1.0] section, and its footer linked releases/tag/v0.1.0 — while git tag -l was empty, no GitHub release existed, and PyPI returned 404. The release workflow fires only on a v* tag, so nothing had ever been built or published. Lesson: a version number is a claim, and a claim in a CHANGELOG is the easiest kind to believe. Verify a release the way a stranger would — git tag -l, gh release list, the package index — never by reading the file that asserts it.

A docs-only merge turned main red. ruff format --check . formats Python fenced blocks inside Markdown, so an igraph snippet in a research doc failed the Format gate. The instinct that a documentation change cannot break the build is wrong in this repo, and the gate is the only arbiter. Now stated in CLAUDE.md and in the README's Development section.

Merging from inside the feature worktree silently does nothing, and the tag lands off-main. Running git merge --ff-only <branch> while cwd is that branch's own worktree merges the branch into itself — "Already up to date" — and the subsequent git push origin main reports "Everything up-to-date" because the local main ref never moved. The git tag that followed pointed at a commit reachable only from the branch, so v0.1.2 existed, was pushed, and was not an ancestor of main. Both commands succeeded; nothing failed loudly. Lesson: merge from the primary checkout, and before creating a release assert the lineage — git merge-base --is-ancestor vX.Y.Z main.

The README was the only surface that lied. An audit against the running CLI found four claims contradicting the code: pnk ask --deep described as existing (it is a v0.4 plan), a budget ledger described as tracking spend (nothing writes one), install lines pointing at a PyPI package that 404s, and the headline KB diagram built on a .pdf — the one file type v0.1 cannot ingest. cli.py and the CHANGELOG were scrupulous in the same places, both saying "planned for v0.4". Lesson: prose drifts toward the design and away from the build, because the design is what the author is thinking about. The check that works is running the commands the README shows. The documented [light] install had the same shape: co-equal in the README, broken at the first sync because pnk init always stamps sentence-transformers.

A promise in a section with no increment number belongs to nobody. plans/20260725_1317-v0.1.md asked for a CI grep gate keeping paid-API clients out of src/ under "Verification of the whole". No increment owned it, so it never shipped — while the invariant it guards is the one CLAUDE.md calls non-negotiable. Now enforced, and verified in both directions: it passes on the current source and catches a planted import openai. Lesson: every promised check carries an increment number and a path, or it is a wish.

Planning v0.2 (20260727 17:00)

A review pass is a change, and a change needs its own review. Adversarial pass 2 over plans/20260727_1543-v0.2.md returned 5 HIGH — and four of the five were created by pass 1's own fixes, not survivals of pass 1's findings. Pass 1 correctly rejected a per-page cost estimate as an order-of-magnitude under-reservation against whole-document requests; the shape it introduced instead was quadratic in input and stopped fitting the model's context window at ~166 pages, so a 100-page document reserved ~$375 and the feature failed closed with a refusal no user could satisfy. Lesson: never implement from the revision that a review produced. Two passes are the floor for a document of this size, and each pass reviews the previous pass's fixes, not only the original.

A threshold that only exists in tests/ protects nobody. Both fitted floors — the one that triggers paid re-extraction and the one that stops a paid run on an already-healthy PDF — were committed to tests/pdf-corpus/baseline.json, which no wheel installs, while three runtime consumers on a user's own KB depended on them. The fail-closed rule was also stated for one floor and not the other, so the guard against paying to re-extract a healthy PDF was silently disabled everywhere. Lesson: a value a user's runtime reads is package data, and every fail-closed rule is stated and tested once per consumer, not once per document.

An append-only ledger needs a way to say "this never happened". Every paid call wrote a reservation before the call and a reconciliation after, with an unreconciled reservation counted at its reserved amount. A call that raised — timeout, 429, 5xx — therefore left spend that nothing could ever close, in a file CLAUDE.md forbids editing: a handful of transient failures would lock a user out of a monthly budget permanently. Lesson: a two-state protocol over an append-only log needs a third state. Reserve → reconcile or void.

Timestamps were composed instead of read. Four "verified on 20260727 17:34" claims were written at 17:00 — 34 minutes in the future. Session context carries a date but never a time, so any HH:MM not read from the clock is invented, and an invented one lands in the future about half the time. A timestamp exists to say how fresh a verified claim is, so a fabricated one is a false evidence claim rather than a formatting slip. Lesson: run date "+%Y%m%d %H:%M" and paste the result; one call covers a batch of edits. Now a rule in CLAUDE.md.

Planning v0.2, pass 3 (20260727 20:23)

Three review passes, and each one's largest source of new HIGHs was the previous pass's fixes. Pass 2 found that four of its five HIGHs were created by pass 1's repairs; pass 3 found the same of five of its twelve. Reviewing a plan is a change to the plan, and a change needs its own review. Lesson: budget for the review of the review. A plan is not ready because the last pass was clean — it is ready when a pass over the current text is clean.

What broke the cycle was method, not effort. Five of pass 3's twelve HIGHs were found only by checking the plan against src/ and against arithmetic — never by reading the plan, however adversarially. --rebuild discards the index and reads its "before" snapshot from the empty new database (sync.py:231-241), so a guarantee recorded only in index.db was invisible to the one command that most needed it. A trace test asserted "exactly one chunk contains this offset" while chunk.py:239-269 prepends the carried overlap and takes start from the carried piece, so chunk n+1 begins inside chunk n whenever a block splits. A filter test named tags as a column; it is json_each over a NOT NULL DEFAULT '{}' field, so the assertion was true by schema for every row and would have passed on a corpus with no tags at all. A raster gate promised "a moved word must fail" against a threshold its own arithmetic puts about four lines of text away. And the docs sweep enumerated README additions while four sentences already in the README are falsified by the release — the same defect, and the same count, as the audit at 0.1.2. Lesson: a consistency pass cannot find a claim that is internally consistent and externally false. Run three narrow passes instead of one wide one — code-reality (resolve every claim about src/ against the file), arithmetic (recompute every stated number), and promise-ledger (walk every enumerated bound, flag, floor, gate and amendment asking "which increment makes this true, and which test proves it"). Each found HIGHs the wide pass missed.

A threshold you cannot fit yet is not a threshold — ship the metric, defer the loop. Two passes each tried to fit the completeness audit's floor, and each picked a pair that was not the pair it is applied to, because the applied pair needs paid model output that does not exist at fitting time. The second attempt would have fitted over a population including a fixture the reader contract requires to raise, landing the floor near zero and leaving the audit inert on every installed copy. Lesson: when a threshold's correct fitting data cannot exist yet, that is the finding. Ship the measurement, report it, and let the release that produces real data fit the number.

"Bypasses module X" is a claim about an import graph, and it was false. The paid extractor was documented as bypassing layout.py — while calling normalise(), which lives in it. The version constant covering that module was then deliberately excluded from the paid fingerprint, so a whitespace or ligature change would have missed every free cache entry and silently hit every paid one, with the coherence check unable to see it. Lesson: when two consumers share one stage, version that stage separately rather than arguing about which of them "really" runs it.

Planning v0.2, pass 4 — three narrow passes (20260727 20:57)

Splitting one wide review into three narrow ones is what finally found the structural defects. Passes 1–3 were each a thorough adversarial reading, and each one's largest source of new HIGHs was the previous pass's fixes. Pass 4 ran instead as a code-reality pass (resolve every claim about src/ against the file), an arithmetic pass (recompute every stated number), and a promise-ledger pass (walk every enumerated flag, floor, gate, constant and amendment row, asking only "which increment makes this true, and which test proves it"). Together they returned 19 HIGH — and almost none of them were things a fourth general reading would have caught. Lesson: a reviewer reading for coherence finds incoherence. Defects that are internally consistent and externally false need a pass that leaves the document: to the source, to a calculator, or to a checklist. Run those as separate passes, because each is a different kind of attention.

A review pass's own fixes are the highest-risk text in the document. Four of pass 2's five HIGHs came from pass 1's fixes; five of pass 3's twelve came from pass 2's; and two of pass 4's came from inside pass 3's fixes — a pnk budget ordering bug fixed by assigning the edit to an increment that lands earlier, and a missing-amendment-row sweep that added a row for one copy of a sentence and missed two others. The arithmetic pass caught the sharpest case: pass 3 replaced a raster tolerance that could not detect a moved word with a different tolerance that also could not detect a moved word. Lesson: never ship the revision a review produced. Re-review the fixes specifically, and prefer a method that cannot be fooled by the same reasoning that produced them.

"The code already does X" is a claim, and this project keeps getting it wrong. Verified against src/: write_sidecar runs only for newly minted documents, so a plan built on "sync writes the sidecar" had no write path for the case it cared about; pnk init --ci was designed in DESIGN and never built, while an increment was written to modify it; CI installs --extra light and runs the model tests in the check job, so a new matrix leg would triple them rather than skip them; and --index-only, which the hooks run, is contractually forbidden from writing into docs/ at all. Lesson: every plan sentence about existing behaviour carries a file:line, or it is a guess.

A threshold needs enough data to have chosen it. The running-head threshold T was documented as fitted over a stratum of 3-page fixtures — where per-document recurrence can only be 1/3, 2/3 or 1, so every T in (1/3, 2/3] reproduces the corpus identically. The value was real, the fit was not. Lesson: state a fitted threshold's resolution alongside its value. If the corpus cannot distinguish 0.4 from 0.6, "fitted" is a claim the data does not support — enlarge the fixture or call it a chosen constant.

I1 — extras, the extractor seam (20260727 22:28)

An explicit, textual exit criterion is not met just because the surrounding tests feel thorough. The plan's own words were "the filterwarnings probe and both marker predicates get named tests" — the probe got tests; pdf_runnable()/paid_runnable() did not, and nothing caught it until the review grepped for their names and found only their own definitions. Twenty-odd other tests passing made the increment feel covered. Lesson: when a plan states a test-coverage exit criterion in so many words, grep for the named thing before calling the increment done — a feeling of thoroughness is not the same claim as a named test existing.

A synthetic probe path that doesn't mirror the real call site's resolution semantics passes on the easy case and is silently wrong on the common one. doctor._could_match_pdf checked whether sources.include could ever match a PDF by testing patterns against a probe path prefixed with the root's own name ("docs/__pdf_probe__.pdf") — but walk_sources applies each pattern via root.glob(pattern), where root is already resolved, so a pattern is relative to it, never to the KB root. The bug was invisible in-repo because the one test written for it used **/*.pdf, which happens to match regardless of the extra prefix; a bare *.pdf — an equally ordinary manifest — silently reported OK: not installed (no .pdf in include) on a KB that would, in fact, fail its very next pnk sync. Lesson: a synthetic probe is only as good as its fidelity to the real resolution path, and one test shape that happens to tolerate an error is not evidence the error doesn't exist — test the shape closest to the literal documented example, not only the most generic one.

The docs-in-the-same-commit rule missed two files because neither is named a "user-facing surface." DESIGN.md and CLAUDE.md were amended correctly in the same commit that made CI a three-leg matrix; README.md's and the Makefile's own make install comments, both reading "as CI does," were not — the exact class of drift a 0.1.2 audit already caught once for this project (above). Neither file is a flag, a manifest key, or a --help string, so neither felt like it was in scope. Lesson: "describes CI/build behaviour in prose" is as much a user-facing surface as a CLI flag — grep README.md and the Makefile for the thing that changed, not only cli.py and DESIGN.md.

I2 — the synthetic PDF corpus (20260727 23:43)

HIGH — a gate rendered at a resolution that made its own threshold unreachable. The scanned stratum's tolerance is "more than 300 pixels differing by more than 32 levels", derived for a page rastered at 150 dpi. The comparison called page.render(scale=1.0) — pdfium's default of 1 px per point, i.e. 72 dpi — which downsamples the stored 150 dpi image ~2× before diffing. That shrinks the page from 2,105,025 px to 485,316 and a moved word's delta from several hundred pixels to well under a hundred: the gate would have passed exactly the change it exists to catch, while its docstring claimed a 2× margin. The docstring also named A4 (1240×1754) for a corpus that is US Letter throughout, so both factors in the derivation were wrong and they partly cancelled, which is why the number still looked plausible. Lesson: a tolerance is meaningless without the resolution it is measured at — state both, and assert the comparison runs at the fixtures' own. When a derived constant is checked, re-derive it from the code that consumes it, not from the prose that introduced it.

HIGH — textwrap invented a hyphen the ground truth then rendered as a phantom space. textwrap.wrap(..., break_long_words=False) leaves break_on_hyphens=True, so it split the existing compound "spine-out" across two lines; the ground truth joins lines with a space, yielding "spine- out" — a string no correct extractor could ever produce, in the one file whose entire job is to be what a correct extractor produces. The same word appears correctly in 16 other places in the corpus, so the corpus contradicted itself. Lesson: when a helper's default silently edits content, the edit shows up wherever the content is re-joined by a different rule. Hyphenation is exercised deliberately here by fixtures that place their own hyphens; it must never arrive by accident.

HIGH — the soft-hyphen fixture's ground truth dropped the first four words of its own page. The expected text began "cooperation agreement…" while the page reads "The clerk filed the coopera- tion agreement…". Written by hand while thinking about the joined word, not about the page. Every automated check still passed: the fixture had a ground truth, the counts matched, the bytes regenerated. Lesson: nothing in a corpus's own test suite can tell you a ground truth is wrong — only reading it beside the page can. Budget for that reading explicitly; "the tests are green" is not evidence about the one thing tests cannot check.

MEDIUM — two claims with no enforcement, in a file full of enforcement. "Pillow is dev-only, never core, never an extra" was stated in the commit message, in conftest.py and in a test docstring — and adding Pillow to [project.dependencies] left the whole suite green, while the structurally identical claim for pypdfium2/anthropic is tested. Separately, pdf_runnable()'s three-part predicate had a test walking false→false→false→false→true that never turned the corpus clause off once both libraries were on, so deleting that clause entirely still passed. Lesson: for an N-part predicate, assert each part is individually load-bearing by turning it off from the all-true state — a monotonic walk up to true proves only the last flip mattered.

LOW — the same quantity was stated twice, differently, and neither figure was right. The commit message said ~440 KB (that is du's block-rounded disk usage of the whole directory), the CHANGELOG said ~370 KB (unreconstructable), and the budgeted quantity — the PDF bytes test_byte_budget sums — is 266 KiB. Lesson: when a number appears in two documents, both are guesses unless one of them was measured; measure once and paste the same figure.

I3a — extraction core, pure: chars to ordered, de-furnished text (20260728 00:52)

HIGH — column clustering compared each candidate to the wrong reference point, letting drift accumulate past its own threshold. reading_order grouped blocks into columns by comparing each new block's x0 to the last-placed member of the current column, not the column's start. Sorted by x0, each step can individually stay under _COLUMN_GAP while the column's accepted range walks steadily rightward — so a genuine third column, far enough from the first to be its own column, could still merge into the second's cluster one small step at a time. Caught with a reproduction script laying out three real columns and reading the wrong (merged) order back. Lesson: "cluster by gap" needs a fixed anchor — the cluster's start, not its most recent member — or the gap check bounds a single step while saying nothing about total drift.

HIGH — y-band clustering by round() put a hard wall at every half-integer. strip_running_heads grouped running-head candidates into y-bands with round(block.y0). Two renderings of one genuine running head at 750.4 and 750.6 pt — sub-point jitter, far smaller than any real layout difference — round to 750 and 751: two distinct, non-recurring signatures, each individually under the suppression threshold even though the line recurs on every page. Fixed with tolerance-based clustering (shared anchors, abs(y0 - anchor) <= _RUNNING_HEAD_Y_TOLERANCE) matching _LINE_TOLERANCE's own approach elsewhere in the same file. Lesson: round() is a clustering method with an invisible discontinuity at every .5; anything claiming "the same, allowing for rendering jitter" needs a tolerance compare, never a shared rounding function, or the false-boundary cases won't show up until real PDFs hit them.

HIGH — the import-purity test only recognised import X, not from X import Y. _imported_names walked ast.ImportFrom nodes and recorded node.module only. from pinakes.extract import layout — the exact style layout.py itself already uses for its own dependency on ExtractedText — resolves to the module name pinakes.extract, plus the separately recorded name layout; the check for "extract.layout" matched neither, so textpolicy.py could have imported layout.py this way and test_textpolicy_is_pure_and_does_not_import_layout would have stayed green. Fixed by folding ImportFrom.names into fully-qualified names (f"{module}.{alias.name}") alongside the bare module. Lesson: an import-graph test written against ast.Import habits misses ast.ImportFrom entirely unless it's built and then attacked with the exact style the file under test itself uses.

MEDIUM — a page's dominant font size was voted on by character, not by line, so a verbose heading could out-vote the body size it was meant to be measured against. _mode_font_size originally took one entry per character in blocks_from_chars; a short body line has few characters, a heading with a long title has many, and counting per-character let a sufficiently wordy heading tip the "mode" size to its own, inverting line_size > body_size for the very line it should have flagged as a heading. Fixed to take one entry per line ([max(c.font_size for c in line) for line in lines]). Lesson: "most common value" needs its unit stated explicitly — voting by the wrong unit of measurement produces a plausible-looking answer that is wrong in exactly the cases with more text, which are also the cases most likely to be headings.

MEDIUM — a symmetric rule was checked on one side only, twice, in different functions. join_hyphenation skipped a block as a join source when it was suppressed, but not when it was itself a heading — so a heading ending in a hyphen could be joined into as a source, even though the same function already refuses to join into a heading as a continuation. Separately, assemble silently produced a truncated document if a block's page_index ever fell outside range(len(pages)) — a caller bug that should be loud (I3b's future pdfium adapter is the only caller that will ever construct page_index), rather than a quietly shorter ExtractedText with no error at all. Both fixed in the same pass: the heading check now runs both ways, and assemble raises RuntimeError on an unplaced block. Lesson: when a function enforces "never X across a boundary," check both directions explicitly — a docstring that says "either side" is a claim, not a guarantee, until both sides have a test — and prefer a loud failure over a silently smaller correct-shaped result whenever "silently smaller" is a shape invariants alone can't distinguish from correct.

LOW — fixing the "no filesystem access" gap introduced a fragile substring check, caught before it shipped. Extending the import-purity test to also assert no os/pathlib/io import used the same marker in name substring style already used for PDF libraries ("pypdfium2" in name). Re-deriving what that check would actually match against layout.py's real imports first — rather than trusting that it passed — showed "io" in name matches typing.Optional and collections.abc.Iterable (...t-i-o-n...), neither of which touches a filesystem; the check would have false-positived the moment either was ever imported. Fixed to match on the module boundary (name == module or name.startswith(f"{module}.")) before it was ever committed. Lesson: two-letter module names are not safe substring needles — os/io collide with ordinary English inside almost any longer identifier — so "does this file import X" must match on the dotted-name boundary, never bare containment.

I3b — the pypdfium2 adapter, extraction-quality metrics, and the two fitted floors (20260728 03:06)

HIGH — an empty page list is not an empty request; it is pypdfium2's spelling of "every page." slice_pages(path, first, last) clamped last to the document's own last page but never validated that first still fell before it. Whenever first > last after clamping — a reversed range, or a first entirely beyond the document — range(first, last_clamped + 1) is empty, and an empty list is falsy in Python: pypdfium2's own import_pages treats a falsy pages argument identically to pages=None, its own spelling of "import every page." Verified directly against the real 12-page baseline-12p.pdf: slice_pages(5, 2), slice_pages(100, 200) and slice_pages(20, 30) each silently returned all 12 pages, no exception. slice_pages is stated as I7b's future paid-path request unit; a future off-by-one computing a page window (the last window of a document whose length isn't a multiple of the window size is the obvious candidate) would have silently sent the entire document to a paid API instead of a small slice — a cost-control failure, not merely a wrong answer, in a project whose one hard invariant is that the free path stays free and spending is never accidental. Fixed with explicit validation (first >= 0, first <= last_clamped) before the range is ever built, raising ValueError. Lesson: an empty collection is not automatically "no items requested" to the function receiving it — some APIs (documented or not) treat empty/None identically as "unfiltered," and a range-clamping function must validate the range is still non-empty itself, not merely non-negative.

MEDIUM — "wide relative to the page" and "spans multiple columns" are not the same fact, and only one of them is safe to test for. reading_order's spanning-block detection was measured against exactly one fixture (two-column-b.pdf's caption, 79% of the page's content span, against a 42% maximum for any genuine column line) and shipped as a fixed fraction, _SPANNING_WIDTH_FRACTION = 0.6. An independently-constructed asymmetric layout — a narrow sidebar beside a much wider main column, a real and common shape, not a contrived one — put the main column's own lines at 77% of the page's content span with nothing in it actually overlapping the sidebar at all; the width-fraction test misread every line of the wider column as spanning and interleaved the two columns line by line, silently, with no error. Fixed by replacing the global-width test with a geometric one: a block is spanning only if its own x1 reaches at or past the next column's own x0 — genuinely bridging into that column's territory, which the caption does (its x1 passes the right column's x0) and the wide sidebar-adjacent column does not (there is nothing to its own right to reach into). Both the original caption case and the new asymmetric-column case are now committed regression tests. Lesson: a measurement taken from one fixture is a fact about that fixture, not evidence the derived threshold generalises — check whether the underlying mechanism the threshold approximates (here, "does this block's own geometry actually overlap another column's") can be tested directly instead, before shipping the approximation.

MEDIUM — a fitting function that raises on a missing upper bound but silently guesses at a missing lower bound is not applying one policy, it is applying two, only one of which is stated. fit_running_head_threshold raised loudly when no true-positive recurrence was ever observed, but silently fell back to max_true_negative = 0.0 when no true-negative was ever observed — and a lower fallback threshold makes strip_running_heads more aggressive, so this fallback was assuming the best case (no decoy content ever recurs) with no evidence for it, dormant only because the current corpus happens to have 76 true negatives. Made symmetric: both empty cases now raise. Refactored the pure midpoint arithmetic out of the corpus-walking function (threshold_from_fractions, taking fraction lists directly) specifically so both raise paths are covered by a direct unit test, not only reachable in principle through a synthetic corpus directory. Lesson: check every "if empty, fall back to X" for whether X is a measured true value (0.0 non-whitespace characters is the true yield of a page with no native text layer — the sibling floor's own fallback, left alone) or merely a plausible-sounding guess standing in for missing data — only the former is safe to leave silent.

MEDIUM — every extraction-quality metric whitespace-flattens its input by design, which means a duplicated-newline regression is invisible to the very gate meant to catch regressions. The \r/\n-character fix (dropping embedded line-break characters that were duplicating assemble()'s own inter-block separator) shipped with zero regression coverage anywhere in the suite: reverting it and re-running every test file, plus the real make pdf-eval gate end to end, produced zero failures, because score_document's own documented design whitespace-flattens both extraction and ground truth before scoring any of the five metrics. Added a structural test asserting the raw, unflattened extraction contains no "\n\n" — verified to actually fail against the reverted code before being trusted. Lesson: "every metric flattens whitespace by design" is a correct, deliberate choice for what those metrics should measure, and also a standing blind spot for anything whose only symptom is whitespace — a fix in that category needs its own structural test, in a file that doesn't flatten, or it ships permanently unguarded.

LOW, bundled — three findings from the same review, each small alone. slice_pages with a negative first leaked a raw pdfium.PdfiumError instead of the module's own ExtractionError (resolved by the same upfront validation as the HIGH finding above, which now catches it before pdfium is ever reached). test_check_script.py's guard test asserted three substrings existed somewhere in check.sh, which stayed green even after deliberately replacing the real make pdf-eval call with a no-op while leaving the explanatory comment above it untouched — rewritten to match the actual if/then/else/fi block and assert where each string falls (inside then, absent from else), verified to fail against the gutted version before being trusted. Rate's numerator/denominator were typed float though every call site produces an int (character, word, and pair counts, and sums of the same) — corrected to int, the type they actually are, with no behavioural effect found. Lesson, shared: "does this test still pass if I break the thing it claims to guard" is a cheap, five-minute check worth running on every new test before trusting it — two of these three would have shipped a false sense of coverage without it.

Cross-platform scanned-fixture rendering — main CI-red since I2, unnoticed for three pushes (20260728 08:13)

HIGH — ./check.sh passing locally was silently substituted for "CI is green," and nobody was checking which one had actually been verified. I2's merge, I3a's merge, and I3b's merge each pushed to main believing the suite was green, because each one was green — on macOS, the only platform any of them had run on. gh run list (prompted by the user asking to check GitHub Actions, not by any check of my own) showed all three runs had actually failed on check (light pdf) / check (light pdf claude), with the identical signature every time: test_scanned_regeneration_within_tolerance — scanned-clean: 8006 pixels differ by >32 levels. Three consecutive merges landed on a red main and every one of them was reported to the user as successfully shipped. Lesson: "the local gate is green" and "CI is green" are different claims — one is evidence for the other, not a substitute for it, and the project's own standing rule (check actual CI status before building on top of a branch) applies with equal force to checking it after pushing to one.

Root cause, confirmed empirically rather than assumed. Every text fixture referenced /BaseFont /Helvetica with no embedded font program (pdfwriter.py, since I2), relying on the PDF reader's own substitution for a font it doesn't have. pypdfium2 ships platform-specific prebuilt binaries; macOS has a real Helvetica installed, ubuntu-latest does not, so pdfium substitutes a different font on each — metrically compatible (identical word-wrap, identical line breaks) but with different glyph outlines. The scanned stratum rasterizes baseline-12p through pdfium at fixture-generation time, baking whichever platform generated it into the committed PDF, so CI's own regeneration (on a different platform) could never match. Confirmed, not theorized: a Docker ubuntu:24.04 container reproduced CI's exact number (8,006 px) on the first attempt, and a diff heatmap of the two renders showed every changed pixel sitting exactly on a glyph edge — same text, same word positions, same line breaks, different anti-aliasing. Measuring all ten scanned pages found cross-platform noise ranging 507-8,262 px depending on how much text and how much the contrast reduction already suppressed it (scanned-low-contrast came in far lower than scanned-clean, consistent with the tolerance test's own documented noise-floor behavior).

The obvious fix (raise the tolerance) was rejected on evidence, not instinct. The measured noise ceiling (8,262 px) sits far closer to the test's stated detection target — a single moved word, "a small fraction of a page" per the test's own docstring — than to the documented whole-page-shift signal (33,451 px) the 300 px threshold was originally sized against. Raising MAX_CHANGED_PIXELS above the noise ceiling would have unblocked CI immediately, but a real single-word regression is plausibly smaller than 8,262 px, meaning a tolerance wide enough to absorb cross-platform noise would very likely also have absorbed exactly the class of regression this test exists to catch — silently, with no way to tell from a green gate. Presented three options (raise the tolerance / scope the test to one platform / embed a real font) with honest pros and cons rather than picking unilaterally, since it's a public-repo, shared-CI-affecting change to a previously "verified" threshold. Fixed at the root instead: pdfwriter.py now embeds a subsetted TrueType font (tests/pdf-corpus/fonts/LiberationSans-Subset.ttf, SIL OFL 1.1 — Liberation Sans specifically for its Helvetica/Arial metric compatibility, so none of generate.py's hand-placed coordinates needed to change) instead of a bare base-14 name, so every platform rasterizes the same outlines regardless of what fonts it has installed. Re-running the identical Docker reproduction after the fix measured 0 pixels changed across every scanned page — not merely under tolerance, bit-for-bit identical. Lesson: when a tolerance-based test's noise floor turns out to be closer to its detection target than expected, re-measure what the tolerance would need to become and check that against the smallest real regression it's meant to catch, before touching the number — a gate that still passes is not evidence it still catches anything.

MEDIUM, from independent review — the fix itself shipped with no regression test. An adversarial re-verification (independently reproducing the pre-fix failure and the post-fix 0-pixel result in its own Docker run, and re-deriving the subset font byte-for-byte from a genuine Debian package to confirm tests/pdf-corpus/fonts/README.md's recipe is real) found that nothing in the diff would have caught a future revert to bare base-14 fonts: the only guard was the pre-existing tolerance test, whose whole failure mode is passing locally and only failing on a second platform — exactly what let this bug live unnoticed for three merges. Fixed with a platform-independent test (test_text_fixtures_embed_a_font_program) asserting every non-scanned fixture's committed bytes contain /FontFile2 and never a bare /BaseFont /Helvetica — verified to actually fail against 89d4fb5's committed fixtures before being trusted, and deliberately written with no pypdfium2 dependency so it runs even on the [light]-only CI leg. The same pass found _font_widths' first_char hardcoded at 0x20 with no symmetric downward extension (unlike last_char), which would have indexed _ASCII_WIDTHS negatively for a hypothetical differences code below 0x20 — unreachable today (grepped every Font(...) call site) but fixed to be symmetric regardless. Lesson: "the existing test would have caught this eventually, on some other platform" is not the same claim as "this increment shipped its own regression test" — a fix for a bug that was invisible on one platform needs a check that doesn't depend on which platform runs it.

I4 — the extraction cache (20260728 10:28)

MEDIUM — a filename collision between a real cache entry and its own in-flight write. Every scanning function (survey, total_stats, clear_all) globs *.json; _write's atomic-write temp file was originally suffixed .json too (.tmp-<random>.json, meant to land beside the final name before os.replace). Verified directly: pathlib.Path.glob("*.json") matches dot-files, unlike shell globbing, so that temp name was scanned as a real entry by every one of them. The window only opens on an uncatchable kill (SIGKILL, OOM, power loss — _write's own except BaseException already cleans up anything else), but inside it a stray file could be double-counted in pnk doctor's totals forever (unreachable via the keyed entry_path() lookup, so a fresh real entry gets written alongside it, never replacing it) and, worse, misclassified as a paid orphan if the abandoned write happened to carry an operation_id — read by the one thing this module exists to protect. Fixed by suffixing the temp file .tmp, never .json, so it can never match the glob regardless of the leading-dot question. Lesson: when several functions all key off "does the filename match this pattern," a temp/staging file used by the same module needs its own, deliberately non-matching pattern — matching the final name's extension by habit (.json in, .json out) is exactly how a temp file becomes indistinguishable from a real one.

LOW-MEDIUM, no live trigger today — a JSON round-trip validated key structure but not value types. _read's reconstruction of per_page_provenance did dict(page) for page in provenance with no check that a page's values were actually strings, unlike page_spans three lines above (int(span[0]), int(span[1])) — the same rigor wasn't applied to both fields reading from the same untrusted JSON. Verified directly: a hand-written entry with {"confidence": None} was accepted as a clean cache hit, silently degrading ExtractedText.per_page_provenance's declared Mapping[str, str]. No live writer currently populates a non-empty per_page_provenance (pdfium.py and the fake backend both rely on the dataclass default), so this was unreachable in practice — but the cache exists precisely to survive untrusted/older/hand-edited files, and I5/I7b are exactly the increments that will start writing real provenance. Fixed with _string_mapping, validating every key and value before trusting the entry. Lesson: "no code currently writes the bad shape" is a fact about today's callers, not a property of the format — a cache that reads its own JSON back should validate every field it reconstructs the same way, not just the ones a current caller happens to populate.

LOW — two verified-true claims shipped with no regression test. A cache-write failure (chmod 0o500 on the cache directory, reproduced directly) correctly returns the extraction result without raising, matching the contextlib.suppress(OSError) comment's claim — but nothing asserted it. Two documents sharing one content_hash within a single KB correctly keep their shared cache entry after one of them is deleted (eviction keys on the hash, which is still claimed by the survivor) — only the cross-KB duplicate case had a test. Both were already correct; both now have a test, one of which (the write-failure case) was verified to actually exercise the except OSError branch by checking no cache file was created afterward, not merely that no exception propagated. Lesson: an already-true claim without a test is one refactor away from becoming a false one with no signal — "this works" and "this is tested" are different sentences even when both are honestly true today.

Reviewed and found correct, not a defect: pnk sync --clear-cache (bare, no argument) deleting paid cache entries along with everything else is the intended I4 behavior, not a gap — the plan's own text ("removed only by an explicit --clear-cache=paid") describes a narrower, paid-preserving variant that lands with I7c's ledger reader, which can price what it would be destroying; building selective removal before that reader exists would mean guessing at a cost nothing can yet compute. Confirmed live: an injected paid orphan was removed by pnk sync --clear-cache --yes along with every other entry, consistent with clear_all's own docstring and docs/DESIGN.md's description.

I5 — PDF chunking, page provenance, and a backend-aware sync (20260728 13:59)

CRITICAL — decision 9's "never silently downgraded" guarantee held for a same-path sync and for --rebuild, and silently did not for everything in between: a rename, or a document's first sync on a machine that never extracted it. The initial design protected a paid extraction two ways: pairing.py's own comparison when a document keeps its path, and --rebuild's copy-forward from the old index. Neither covers Adopt/Rename outside a rebuild, which instead fell through to _extract_for_index's only remaining signal — an extract/cache.py hit. A cache miss was then read as "content changed", which is a false equivalence: a --clear-cache immediately before a rename, or the first sync of a KB whose paid PDFs were extracted on a different machine (docs/ committed, .pinakes/ gitignored — the ordinary shape of a clone), miss the cache identically without the file having changed at all. Confirmed live, both ways: (1) paid-index a PDF, --clear-cache, rename it with its sidecar travelling, sync — landed a PaidExtractionRequiredError falsely claiming changed content, and left the index describing the old, now-nonexistent path, since the failed transaction rolled back before the rename could be recorded; (2) paid-index a PDF, rm -rf .pinakes (index and cache, simulating a fresh clone), first sync — identical false failure on a document that had never changed at all. An independent adversarial review (a fresh subagent, unanchored on this increment's own design reasoning) found and reproduced both.

Fixed by moving the "has this changed" decision off the cache entirely: the sidecar's own provenance.extraction gained a fourth field, content_hash — the file's hash at the moment of that specific paid extraction, distinct from the general change-detection hash docs/DESIGN.md §2.2 already refuses to store, since this one changes only when a fresh paid extraction runs. Comparing it directly against the current file's hash answers "changed?" without consulting any cache or index at all. Getting the actual text without paying again is a separate question, now answered in a fixed order: this same sync's own connection (covers a rename — the row is still there under its old path), the old index during a --rebuild (unchanged from the original design, now keyed on doc_id rather than content_hash+path, since a rename's action only ever carries its current path and the old index's row still has the old one), then extract/cache.py. Only when none of the three has an answer does it fail — and now with a new, distinct error (PaidExtractionUnavailableError) naming the file unchanged but its text simply not present on this machine, never conflated with "content changed" (PaidExtractionRequiredError) again. The cross-machine case (2) is not solved by this — there is genuinely no durable, shared home for a paid extraction's text yet — but the failure is now honest about which situation it is, which docs/DESIGN.md §9 records as an accepted, disclosed limitation rather than a silently discovered one. Lesson: a mechanism justified as "the answer must not depend on the cache" (the --rebuild + --clear-cache case this increment was explicitly designed around) needs that property checked against every path that can reach the same decision, not just the one path the design started from — pairing.py's own decision table has four ways into "paid, free-effective, unchanged", and only one of them was ever traced all the way through.

HIGH — --rebuild turned "a paid document's content changed" into "the document vanishes from the index", where a normal sync leaves it stale but searchable. pair()'s PaidExtractionRequired action (decision 14) can only ever fire against a populated before, so it can never fire during a rebuild at all (before is empty by construction) — meaning a changed-hash paid document reaching --rebuild fell through to _extract_for_index's ordinary raise, caught by _apply's generic exception handler, which never inserted a row for it in the first place. Confirmed live: paid-index a PDF, change its bytes, pnk sync --rebuild — report.ok correctly False, but the document was gone from the rebuilt index entirely (zero chunks), not merely flagged. A normal (non-rebuild) sync hitting the identical case leaves the old row, chunks and embeddings untouched instead. Fixed by extending the rebuild copy-forward to the changed-hash case too: the stale row is copied forward at its old content_hash regardless of whether the current file still matches, and a failures entry is recorded only when it does not — so a rebuild now reaches the same outcome a normal sync already gives this exact case, rather than a harsher one purely as a side effect of which command happened to run. Lesson: --rebuild is documented as "free, deterministic, cron-safe" — exactly the description that invites treating it as interchangeable with a normal sync. Any new failure mode this increment gives normal sync a considered answer for needs the identical question asked of --rebuild explicitly, since its empty before makes "the same case" arrive by a structurally different path that is easy to reason about in isolation and forget to reconcile.

MEDIUM — a fresh paid-provenance write left the very next sync one RefreshMetadata cycle away from settling. _index_document decides sidecar_hash from the walk, before it rewrites the sidecar with fresh provenance.extraction a few lines later in the same call — so the hash it writes into documents.sidecar_hash was already stale the moment the write happened. Confirmed live: three consecutive syncs of a freshly paid-extracted PDF produced embedded=1, then refreshed=1 (unexpected — nothing the user did changed), then finally skipped=1. Fixed by recomputing sidecar_hash from the file just written, whenever a write happened, before it reaches the documents INSERT. Lesson: any function that both decides a hash-like value from an earlier read and has the power to invalidate that exact read later in its own body needs the value recomputed after the write, not carried from before it — "I already have this value" stops being true the moment code between the read and its use can change what it should have been.

Caught live, during the adversarial review itself, not by this increment's own author: two different documents can share one content_hash with only one of them ever paid-extracted (a second document minted later for identical bytes gets its own ordinary free extraction). The original rebuild copy-forward keyed its survivor lookup on content_hash alone, which would have let the free twin's own rebuild incorrectly inherit the paid one's chunks, embeddings and backend label. Found and fixed inline by the reviewing subagent (re-keying on (content_hash, path), later superseded by the doc_id keying above once the rename/clone findings required a broader redesign), with its own regression test. Lesson: a value that is "usually unique in practice" is not a key — content_hash was never meant to identify one document, only to detect whether one had changed; this table's actual primary key was sitting right there the whole time.

I6a — budget core, pure (20260728 17:52)

HIGH — every test of the timezone conversion that is this module's entire reason to exist would have passed with the conversion deleted. window.py aggregates ledger records into day/month totals in [budget] timezone, converting both now and each record's reserved_at before comparing. Every test — including the midnight, month-end and DST-transition trio written specifically to exercise attribution — constructed both values already in the target zone, where .astimezone() is a no-op. The adversarial review mutated local_now = now.astimezone(timezone) to local_now = now, and separately the per-record conversion, and both mutations passed all 35 tests. The real case is ordinary, not exotic: a ledger storing UTC timestamps (the obvious choice for I6b) read under a non-UTC [budget] timezone — 23:30 UTC on the 15th is 00:30 on the 16th in Berlin, so either mutation silently files the spend under the wrong day. Fixed by adding a test that aggregates a UTC-stamped record against a Berlin window. Lesson: a test whose fixture is built in the same units the code converts to cannot detect a missing conversion. Exercising a transformation requires input where the transformation actually changes something — three tests that carefully varied the clock while holding the timezone constant proved nothing about the timezone at all.

MEDIUM — an exception hierarchy copied from a sibling module inherited the wrong exception types. prices.py was deliberately modelled on extract/floors.py, including its except (TOMLDecodeError, KeyError, TypeError, ValueError) around parsing. But floors.py parses with float(x), which raises ValueError on bad input, while prices.py parses with Decimal(str(x)), which raises decimal.InvalidOperation — not a ValueError subclass (InvalidOperation → DecimalException → ArithmeticError). A single-character price typo (a European "5,00", an unfilled "TBD") therefore escaped as a bare InvalidOperation instead of the named PricesMissingError the module's own docstring promises, and the test claiming to cover this only ever exercised a TOML syntax error. Lesson: when mirroring a module's error handling, the except clause travels with the parsing call it was written for. Changing float to Decimal changes which exceptions are possible, and an inherited except tuple is a claim about the old code.

MEDIUM — validation absent at the one boundary where a wrong sign inverts the guarantee. estimate_document accepted any pages/pages_estimated: pages=0 produced requests=0 and made per_request_eur raise on a zero division, and a negative pages_estimated produced a negative total_eur. Every other failure mode in the module — unknown model, missing prices, stale prices, oversize request — had a named error, but the one that would make a budget guard understate spend had none. Not reachable from any caller in this increment, since nothing calls it yet; guarded at the source rather than trusting a caller that does not exist to be written correctly. Lesson: for a component whose whole job is bounding a number, the input validation that matters most is whichever one lets the number move in the safe-looking direction.

LOW, worth keeping — three assertions that were true but untested, in the same shape. reserve_document's "every blocked window is named" was only ever tested with all three windows breaching at once (so "always names all three" would also have passed); reserve's "first breach in order wins" was only tested where a single window could breach (two of three caps were set to a generous 100 in every case); and confirm_above_eur's strict > boundary was asserted only incidentally. All three were verified correct by hand and none was a defect — but a regression in any would have been invisible. Lesson: a passing test suite says nothing about the claims it never puts under tension; the boundary cases worth writing are the ones where two plausible implementations disagree.

Also fixed: Table.decimal()'s default path returned before its own minimum check, so a below-minimum default would pass silently — integer()/number() avoid this for free by sharing one code path between default and parsed value, which is why the bug was invisible by analogy. ContextWindowExceededError's remedy told the user to lower a [chunking]-equivalent slice size that does not exist as configuration (K is a fixed constant). Every fix in this increment was confirmed to fail against the pre-fix code before landing.

I7a — the paid-path allowlist gate (20260728 19:25)

HIGH — the gate found two live paid-client imports on the free path, in code that had shipped. doctor._extraction reported whether the configured extractor was available by calling load_extractor(backend), and sync._missing_pdf_extra did the same to decide whether a skipped .pdf still needed an extra. The registry's factory imports the client (extract/__init__.py's _import), so on a KB whose [extraction] backend is claude-vision, both pnk doctor and pnk sync pulled anthropic into a free-path process — doctor on every run. Neither was reachable from any test, because every test KB is configured for pypdfium2. Nothing could actually spend (the extractor is an I7b stub), so the cost was an import rather than a charge — but the invariant CLAUDE.md calls non-negotiable was already false when the gate arrived to check it. Lesson: "does the free path import a paid client" is a question about a running process, not about the source text. A grep over src/ was green the whole time both leaks existed, because neither leak is an import statement — it is an ordinary function call that reaches one.

HIGH — the gate's own paid KB was silently never paid. The free-path runner configures a second KB for claude-vision, since that is the only configuration where the two probes above fire. It did so with text.replace('backend = "pypdfium2"', ...) against a manifest template that has no [extraction] section at all — so the replace matched nothing, the KB stayed on the free backend, and the gate would have passed whether or not the leaks existed. Caught only because the run printed pdf extractor: pypdfium2 importable for a KB that was supposed to say claude-vision. Fixed by appending the section rather than replacing, by making every manifest rewrite a _replace_once that raises when it matches nothing, and by loading the manifest back through the real parser and asserting extraction.backend is what was asked for. Lesson: str.replace returns the string unchanged when it matches nothing and reports it to no one — the perfect way to build a test fixture that does not test what its name says. A fixture whose whole purpose is to be in an unusual state must be read back through the parser and checked.

MEDIUM — two of the first six mutations survived, and both survivals were the finding. Mutating is_backend_installed to return True passed the entire suite: its only two callers are tested with it monkeypatched, so nothing exercised the function itself. And test_a_directory_entry_fails_gate_1 asserted only a non-zero exit — which gate 2 produced anyway by reporting the planted import, so the test stayed green with gate 1's directory branch deleted. Fixed with three direct tests (including one whose probe module raises on import, so a regression to importing errors rather than fails quietly) and by asserting gate 1's specific message. Lesson: a function whose every caller stubs it out has no test, only agreement; and an assertion on an exit code cannot tell which of two gates produced it.

MEDIUM — the runtime checker would have flagged google.protobuf as a paid client. Matching sys.modules names by root (name.split(".")[0] in {"anthropic", …, "google"}) makes every google.* module a hit, and protobuf arrives transitively with onnxruntime, grpc and much of the ML ecosystem. Not triggered on this platform today — verified, zero google.* modules in a free-path run — which is exactly what makes it dangerous: it would have fired first on some other CI leg, on a change having nothing to do with money, and the obvious repair for a safety gate that cries wolf is to weaken the gate. Now matched on a dotted-prefix boundary against google.generativeai in full. Lesson: for a gate whose failure mode is "someone turns it off", the false-positive direction is as load-bearing as the true-positive one — and a latent false positive is worse than an active one, because it lands on an unrelated change.

LOW, worth keeping — gate 2 cannot see a dynamic import, and that is deliberate. extract/__init__.py calls __import__("anthropic", …) inside the claude-vision factory, which no import-statement grep will ever match, and it is not on the allowlist. Exempting the registry would exempt the one file where an accidental static import is most likely; the dynamic call only runs when a caller has explicitly selected a paid backend, which is an allowlisted entry point. The limit is written into the gate's own docstring, and gate 4 is what covers the direction gate 2 cannot: no spelling of an import hides from sys.modules. Recorded so the next reader does not "fix" gate 2 by widening the allowlist.

Also: pnk doctor no longer proves a paid backend's adapter constructs, only that its library is locatable — the necessary price of not importing it, and worth remembering at I7b, when _load_claude_vision stops being a stub. Every fix above was confirmed to fail against the pre-fix code: 10 mutations planted, 10 detected, including both fixes made during this review.

I6b — budget I/O: the ledger, pnk budget, hooks that cannot spend (20260728 23:27)

HIGH — the accountant handed out a PaidCall object, undoing the one guarantee the ledger module exists to provide. budget/ledger.py's whole argument is that a void may only be written when no response was received, and that this is enforced by paid_call being a context manager rather than a convention someone remembers. Accountant.open_call then wrote the reservation and returned the object, putting both the void/unknown decision and the closing write back in the caller's hands — and the caller it was written for is I7b's retry loop, the most branch-heavy code in the release. Its own test left a permanent unknown outcome behind and asserted nothing about it. Now a context manager delegating to ledger.paid_call. Lesson: an invariant enforced by a control-flow construct is only enforced where that construct is actually used. A convenience wrapper one layer up is exactly where it gets quietly opted out of, and "the module below guarantees it" stops being true the moment a caller can hold the handle.

MEDIUM — a single bad character in the ledger could take pnk budget down entirely. Every euro figure is cost_usd / usd_per_eur, computed in a property called long after parsing, from inside the summing loop. A line with usd_per_eur of "0" therefore raised DivisionByZero out of a read-only reporting command — defeating the malformed-line counting whose entire purpose is that no one bad line can do that. The parse-time checks covered type (a JSON number is refused, so no float gets in) but not domain. Rates are now validated positive at parse time, where a failure is a counted malformed line. Lesson: validation placed at the parse boundary only protects what is computed at the parse boundary. A derived value computed lazily elsewhere needs its inputs checked where they enter, not where they are used.

MEDIUM — fsync on the file does not make the file's name durable. The reservation is written before the call precisely so a crash during the call cannot lose it, and each write is fsynced. But creating a file and fsyncing its contents leaves the directory entry unsynced, so the very first reservation a KB ever writes — the one before its first paid call — could vanish on a crash while every later one survived. The parent directory is now fsynced on creation only. Lesson: durability claims have to name what is durable. "The write is fsynced" is a claim about bytes; "the record survives a crash" additionally requires the file to still have a name.

MEDIUM — both hook checks read root/.git/hooks directly, so they were blind in a git worktree. Inside a worktree or submodule .git is a file pointing elsewhere; hooks.hooks_dir has resolved that since I12, and doctor never used it. Every hook read as absent, so pnk doctor reported "0 of 3 installed" and I6b's new machine-driven-spend check would have reported "no hooks installed, so no automatic sync runs" on a KB whose hooks were installed and running. Worth more than its size here: this project's own CLAUDE.md mandates a worktree for every change, so the layout the check is blind on is the one it is developed in. Lesson: a helper that already handles a case is not the same as using it — the second reader of .git reintroduced the bug the first one had solved.

LOW, worth keeping — one report, two clocks. pnk budget computes its windows in [budget] timezone and printed the recent-operations list in the machine's local zone, unlabelled. On a KB synced from two machines the same operation renders at two different times, and the day a call is filed under would not match the timestamp beside it. Recorded because the fix is trivial and the class of bug is not: a report that derives one number from configuration and another from ambient state looks consistent on the machine it was written on.

LOW, second pass — two flags that read as the opposite of what they do. pnk budget listed the five most recent operations and stopped, with no line saying how many it had not shown — the silent-cap failure this plan's own ground rules name ("if a workflow bounds coverage, log() what was dropped"). And --clear-cache's bare form parsed to the value free, which reads as "clear only the free entries" when both spellings clear the whole cache; argparse validates const against choices, so a private sentinel is not available and the bare form has to be a real, honest word. It is now all, and the value names what is being authorised rather than what is removed.

LOW, third pass — the notice init --ci printed described a file it had not written. FREE_BACKEND_NOTICE began "hooks run pnk sync --extract=pypdfium2 …" and was printed by two callers: install-hooks, which writes hooks, and init --ci, which writes a workflow. So pnk init --ci announced .github/workflows/pinakes.yml and then explained what the hooks do. Every test asserted the flag and the phrase "can never spend", both of which were present and correct. Found only by running the command the docs tell a user to run — the constant is now subject-less and each caller supplies its own ("each hook …", "it …"). Lesson: a shared string with a subject baked in is correct for exactly one caller. The tests all checked the part that was shared; the part that was wrong was the part no assertion mentioned.

Also: doctor printed cost_eur — a Decimal division — with a bare f-string, putting all 28 significant digits into a health-check line; the --resolve record's operation field was documented as a value it never takes. Every fix above was confirmed to fail against the pre-fix code: 14 mutations planted over the implementation (14 detected), and 5 more reverting each review fix in turn (5 detected), the last of which found that the formatting fix had no test at all until one was written for it.

---

I7b — the paid Claude-vision extractor (20260729 00:24)

HIGH — the reconciliation recorded the reserved amount, which makes the whole reservation/reconciliation protocol a no-op. _billed_call closed each successful call with cost_usd=reserved_eur * usd_per_eur — the estimate again, not what the response said it cost. The shape was perfect: a reservation, then a reconciliation superseding it, exactly as I6b's protocol requires, with a ledger pair per call and every test about pairing passing. What it superseded the reservation with was the reservation. Every window would have charged worst-case forever, pnk budget would have reported an estimate as spend, and the reconciliation record's presence is precisely what would have made it look settled. Fixed with actual_cost_usd, derived from the response's own usage and the model's price. Lesson: I6b's tests could only ever check that a reconciliation exists and supersedes; that it carries the right number is a claim only the increment that produces the number can make. A protocol test and a value test look alike and are not — and every mutation I planted over the retry logic passed straight through this, because the bug was in the one line none of them touched.

HIGH — one bad PDF would have crashed a 1,000-document sync. TransportError and RequestTooLargeError were plain Exceptions. sync isolates each document behind except (PinakesError, OSError, ValueError), so an exhausted 429, a 500, or an oversized page would have escaped that handler and taken the entire run down — the exact opposite of the per-document isolation §6.4 promises and pnk sync's own "one broken PDF cannot block a 1,000-document corpus". Not caught by any test, because every test called extract_slice directly and asserted pytest.raises(TransportError), which passes identically whichever base class it has. Lesson: an exception's type is part of its contract with a caller several layers up, and a test that catches the exception it just raised cannot see that contract at all.

MEDIUM — two mutation survivors, both the finding. A cap check hoisted out of the transport retry loop survived because every attempt inside that loop voids at zero: nothing the loop does moves the total, so the omission looks harmless. It is not — between a 429 and its backoff another process syncing the same KB can spend the headroom, and the retry would go out anyway. And the per-slice semantic budget survived because every test used a single slice, where "per slice" and "per document" are the same number; that was a defect I had found and fixed while writing the loop and then never put under tension. Lesson: a bound that is only ever exercised at N = 1 is not tested, it is agreed with.

MEDIUM — the module imported pypdfium2 at module scope, which §4.4 cannot afford. The fingerprint path reaches this module on every query, on whatever install the user has, so a top-level pdfium import made a coherence check on a claude-vision KB fail outright on a core-only install. Caught by test_coherence_never_imports_a_paid_client — a test written in I2 for a different reason, which happened to be the exact shape of this mistake. Recorded because it is the second time this project has been saved by an import-graph test that nobody wrote for the case that caught them.

LOW, worth keeping — the gate refused the commit, which is the gate working. .paid-path-allowlist shipped empty at I7a specifically so that its first real entry would be earned, and it was: the commit creating claude.py failed until the line was added. Its test then turned out to assert only "0 exempt paths", which would have passed on any addition at all, so it now pins the expected contents — widening an allowlist is how a gate like this one dies. Two dead exception classes (TruncatedResponseError, RefusalError) were also removed: the stop_reason branches replaced them and nothing ever raised either.

LOW, second pass — --estimate-only demanded an API key from a KB with nothing to estimate. The transport was built before the walk, so a KB with no PDFs failed on a missing key instead of reporting nothing. Built on the first PDF now. Also cleaned up in the same pass: an unreachable except RequestTooLargeError around slice_pages (only build_request raises it, and by then the call is committed — the size question belongs before anything is built), and a repeated "claude-vision" literal where the module already imports the constant.

HIGH, fourth pass — the paid fingerprint omitted the model, so changing it reused another model's text. The plan states the inputs as "(backend name, model id, prompt version, schema version, request-shape version…)" and I wrote every one of those except the model. The cache key is <content_hash>-<fingerprint>, so editing [extraction] model would have hit an entry a different model wrote, with no miss, no warning and no stale marker — the §4.4 machinery intact and looking at the wrong key. The plan names two tests for exactly this (test_changing_the_model_misses_the_cache, test_changing_k_misses_the_cache) and I had written a placeholder for the first that only asserted two unrelated fields were non-empty, which is how it stayed invisible through three review passes and fifteen mutations. Fixing it meant threading [extraction] model through the registry's FingerprintInputs contract — two real callers, sync and the §4.4 coherence check — plus a test that the free backend's key is unperturbed by the new parameter, or every existing free KB's index would have gone stale the day it was added. Lesson: a test written to a name from the plan, but not to the claim behind the name, is worse than a missing test — the plan's checklist reads as satisfied. The tell was there in the placeholder's own body: it asserted things that could not fail.

MEDIUM, fifth pass — three of the plan's named tests had no implementation at all. Auditing the plan's test list against the file (prompted by pass 4, since the same failure mode was in play) found the multi-slice document, the oversize-slice split, and the no-floor-installed refusal all missing — the first being where the slice-window arithmetic and the short final slice actually meet a real PDF, and where an off-by-one either drops pages or sends the whole document to a paid API. Writing them found nothing broken, which is the useful outcome to record: the code was right and unwitnessed. Two of them also had to be sized from the fixture rather than from a constant — a hard-coded byte threshold either never splits or recurses straight to the single-page failure, and which one it does is a property of the corpus, not of the code. slice_windows and slice_bytes lost their underscores in the process: they are the two things a reader most wants to check.

LOW, sixth pass — nothing drove pnk sync itself. Every test exercised a piece: the slice loop, the ledger pairing, the cache's join key, the accountant's windows. The pieces were wired together across four modules, and "each part works" is not the claim "the parts are connected" — which is the seam an increment is most likely to get wrong. One end-to-end test now runs a real pnk sync over a paid KB and checks the whole chain: the document indexed under claude-vision, the ledger's call reconciled, and the cache entry carrying the operation_id/call_ids that §6.3 left null until this increment. It passed first time — but writing it introduced the pass's one real defect, and only the [claude] leg saw it: the test swapped the registry entry and then called unregister_extractor, which deletes. There is no undo for a name the package registers at import, so two unrelated tests later in the session lost claude-vision entirely. Fixed with a registered_entry accessor and a re-register in the finally. Lesson: a test that mutates process-global state needs to restore the previous value, not remove its own — and a suite that only ever runs on one CI leg will not show you which.

MEDIUM, seventh pass — the [light] leg's green was partly an artefact, and I had not really run that leg at all. uv run --extra light does not prune extras a previous --extra pdf --extra claude installed, so "all three CI legs pass" was one leg run three times. A real uv sync --frozen --extra light — what CI actually does — then showed the paid suite failing on its own while passing in a full run: tests/test_extract.py leaves a fake pypdfium2 in sys.modules, and two --estimate-only tests were quietly relying on it. The underlying cause was mine: _estimate_only imported page_count before the walk, so a KB with no PDFs demanded the [pdf] extra to be told it had nothing to estimate — the same defect as pass 2's transport, one line above where I had fixed it. Lesson: uv run --extra X is not uv sync --extra X; verifying a matrix means reproducing what the matrix does, not asking for the same set three times. And a suite that only ever runs after its neighbours cannot tell you what it depends on.

LOW, third pass — markdown emphasis reached a terminal. --estimate-only's help text carried **A network call**, which argparse renders as literal asterisks: the emphasis was written for CLI.md and pasted into a surface that has no renderer. Now checked against every command's rendered --help output rather than argparse's internals — the artefact a user actually sees. Backticks are deliberately allowed, since [extraction] backend reads fine in a terminal and is the convention this CLI already used; flagging those too would have been a style crusade over pre-existing text rather than a defect.

Also: stubs/anthropic.pyi joins stubs/pypdfium2.pyi, because the strict type gate runs on the [light] leg where the package is absent. It records the one relationship easy to get wrong from memory — APIConnectionError is a sibling of APIStatusError, and APITimeoutError a subclass of the former — because checking them in the wrong order classifies every timeout as a plain connection failure, which is the difference between recording €0 and admitting a possible charge. 18 mutations planted in total, 18 detected once the survivors got tests.

---

I7c — the completeness audit, staging, all-or-nothing (20260729 02:38)

MEDIUM — three of ten mutations survived, and all three were the same shape: a rule with no test that it fires, only that it exists. Clearing staging after the complete entry is written had no test that staging is ever cleared at all — and stale staging is not litter, because its key is <content_hash>-<fingerprint>: a later run of the same document would find it, skip slices, and serve text from a superseded extraction silently and for free. Stopping the corpus at the first cap breach had no test that drove sync; the on_exceed tests build a SyncReport by hand, so they can check what a stop means but never whether the loop stops. And keeping staging out of the cache root had nothing asserting that survey, total_stats and clear_all cannot see it. Lesson: the tests I wrote were about the rules I was thinking about while writing the code, which is exactly the increment-shaped blind spot this project keeps rediscovering. Mutation found all three in one pass because a mutation asks the question the test forgot: not "is the rule stated" but "would anything notice if it stopped being true".

MEDIUM — the audit's own fixtures measured nothing. word_coverage tokenises on [a-zA-Z]+, so forty words of f"{seed}{index}" collapse to one distinct word: every page scored 1.00, the dropped-content test found no outlier, and the fixture could not have failed whatever the code did. Caught only because that test failed for a different reason first. Lesson: a fixture built for a metric has to survive that metric's own tokeniser — and "the test passes" would have hidden this completely if the assertion had been slightly weaker.

MEDIUM, second pass — the audit made the very mistake it was written to avoid, one branch later. A page above the yield floor whose text holds no significant words — a table of figures, which is an ordinary PDF page, not an exotic one — gives word_coverage a denominator of zero. I scored it 1.0: full preservation claimed for something never checked, and it drags the median up, making the genuine outliers look less unusual. Six lines earlier the module argues at length that a scanned page must be exempt rather than zero, for exactly the same reason in the other direction. Now both are exempt. The fix then exposed a second worthless fixture: _significant_words keeps only words of four characters or more, so prose("a")'s three-letter words were never measurable either — that test had been passing purely on the 1.0 default it was supposed to be testing around. prose now asserts its own output is measurable. Lesson: a default that makes "unmeasurable" indistinguishable from "perfect" hides broken fixtures as effectively as it hides broken code — and I wrote the argument against it into the docstring of the function that did it.

LOW, second pass — one feature, two standards for the same question. _is_budget_refusal identifies a refusal by exception type, with a comment saying an error string is prose and prose gets reworded. Twelve lines later SyncReport.ok decided which failure was the budget's by comparing that same prose. It now matches on the path, which is structural.

LOW — on_exceed had been parsed, validated and read by nothing since v0.1. A manifest key with a choice() validator, a default, a template comment and a documented meaning, wired to no behaviour at all. It now decides whether a budget stop is a failure. Recorded because validation is what made it invisible: a key that round-trips through the parser looks implemented from every angle except the one that matters.

LOW — the interruption I first scripted was not one the caller isolates. The test double raised AssertionError when its script ran out, which sync does not catch, so it escaped the per-document handler and failed the test rather than the document. Replaced with a timeout — a real interruption, and one the caller actually handles. A test double's failure mode is part of the test: if it fails in a way production cannot produce, the path under test never runs.

LOW, and the second time in one session — I composed a timestamp instead of reading one. This entry's heading was written 03:04 moments after date had printed 02:38. CLAUDE.md's rule is already explicit ("read the clock; never compose a timestamp"), and knowing the rule is evidently not the same as running the command: the failure is that composing feels like recalling. Caught both times only by diffing the two numbers on screen.

Also: another agent, working independently, found that I7b's own docs contradicted themselves — STATUS.md said "claude-vision is a real extractor" in one row while the prose eight lines below still explained that nothing can spend because it is a stub. I had updated the table and left the paragraph justifying it. Seven review passes over I7b did not catch it. 15 mutations planted here, 15 detected once the survivors got tests.

---

---

The eval harness: three defects under one green suite (20260729 03:23)

Found while planning the links and graph releases, whose whole gate rests on this harness. All three were live on main and all three passed every test.

HIGH — the multi-hop class measured nothing about hopping. Outcome.hops_followed was computed for every scripted question and read by no metric — not recall_at_k, not by_kind, nothing compare() looks at. Deleting the hop loop outright left by_kind["multi-hop"] bit-identical. A multi-hop question was a single-shot search of its last hop's query wearing a label. The one guard was assert any(outcome.hops_followed > 0 …) — an any() over five questions, on a field that fed nothing.

HIGH — and that hid a defect in the golden set itself. Three of the five questions named their last hop's document in expect; two named their first. So the scorer ran a query about brittle-paper conservation and demanded the annual report. Nothing could catch the disagreement, because hops fed no metric that could notice it. The fix makes expect exactly the union of the hops' documents and asserts it for the committed set.

The numbers moved because the scorer was wrong, not because retrieval changed. recall@5 0.8788 → 0.9091, MRR 0.7737 → 0.8116, rerank precision 0.7273 → 0.7576, by_kind["multi-hop"] 0.80 → 1.00. Stricter scoring, higher score — because the two inverted questions had been asked about the wrong document all along. A metric that improves when you make it stricter is telling you it was measuring something else.

MEDIUM — compare() wrote by_kind into every baseline and never read it back. A change lifting one class and dropping another by the same amount moves the aggregates by almost nothing; CI was green through it. The question count had the same shape: written, never compared, so a golden set that silently lost its hard questions would have scored better.

MEDIUM — the "cheap deterministic embedder" was not deterministic. HashingBackend hashed each word with hash(), which Python randomises per process for str unless PYTHONHASHSEED is set — and nothing sets it, nor can a conftest.py, since the value is read before the interpreter starts. Which words collided in the 64-dimensional space changed run to run: one failure in 40 runs before, zero in 60 after switching to zlib.crc32. It surfaced only because a newly written test tripped over it once. A fake that cannot reproduce itself cannot tell a real regression from its own noise (v0.1 rule 5).

The transferable lesson. All three survived because the tests asserted that the machinery ran, never that it could detect. The mutation pass is what caught them: four mutants — hit ignoring hops, the by_kind comparison, the question-count check, and the golden-set consistency assertion — were introduced deliberately and all four killed a named test. Green proves the tests ran; only breaking the code on purpose proves they can see.

Shared-file contention tooling (20260729 04:06)

HIGH — git status --porcelain's leading space is significant, and a helper doing .strip() on the whole output silently ate it. The overlap gate's git() wrapper returned proc.stdout.strip(), which is correct for merge-base and symbolic-ref and wrong for status --porcelain: a modified file is reported as M CHANGELOG.md with the status in columns 0–1, so stripping the output removed the first line's leading space and the path parsed one character short — HANGELOG.md. It matched nothing, and the gate reported "no overlap" with total confidence. Exactly the one failure a contention gate cannot have.

Two things about how it was caught, both worth keeping:

• The tests drive real git against real temp repositories, not a mocked subprocess. The gate is almost entirely a set of claims about git's behaviour — what diff A...B means, which commit merge-base picks, how status --porcelain spells a rename — and a mock asserts the author's belief about each of those rather than the behaviour. A mocked test would have returned " M CHANGELOG.md" from a fake and passed with the bug present.
• The mutation pass re-introduced this exact bug deliberately and confirmed the right test fails. git() is now documented as trailing-newlines-only, with the reason, because the next person to "tidy" it back to .strip() will find nothing obviously wrong.

MEDIUM — a clean auto-merge is not a correct merge, and only the loud half of that was being managed. Three parallel branches edited CHANGELOG.md, docs/STATUS.md and docs/DESIGN.md inside one hour on 20260729. CHANGELOG.md conflicted and was resolved by hand; the other two merged silently, because the edits landed on different lines. Git merges edits that do not overlap textually, never edits that agree — so two agents can leave one document contradicting itself with every command reporting success, and no conflict resolution however careful would surface it.

The response is deliberately in two layers, because one does not cover the other's cases:

• changelog.d/ and retro.d/ remove the cause for the two documents every change must write to — separate files cannot conflict, so for those the class stops existing.
• tools/shared_file_overlap.py reports what remains, which is the living documents (docs/STATUS.md, docs/DESIGN.md) that fragments do not suit, because they are edited in place rather than appended to.

MEDIUM — splicing produced two ### Added headings under one ## [Unreleased], and only cutting a release revealed it. The tool inserted each rendered ### Category block under the anchor without looking at what was already there, so a section that already carried an ### Added from unmigrated prose ended up with two. Keep a Changelog expects one heading per category, and a reader scanning for "what was added" stops at the first. Fixed by merging into an existing heading when there is one, bounded to the anchor's own section so a shipped release's ### Added is never written into.

Worth keeping for the reason it was missed: test_apply_leaves_existing_unreleased_prose_exactly_ where_it_was was written deliberately, and it passed — leaving existing prose alone is correct. The case it did not imagine is that the existing prose has its own category headings. A test written by the reasoning that wrote the code inherits its assumptions, which is the same increment-shaped blind spot CLAUDE.md already names; here the escape was dogfooding, not mutation testing, because the mutation pass only ever perturbs cases somebody already thought of.

Separately, and pre-existing: [Unreleased] had accumulated seven category headings by hand over several days — two ### Added, three ### Changed, two ### Fixed. Consolidated when cutting 0.3.0, with a check that every non-heading line survived the regrouping.

LOW — the fragment tool takes --repo so its tests can drive the real artifact. Importing a tools/ script from a test needs sys.path surgery that pyright and ty then cannot resolve — ty failed the build on exactly that. Running it as a subprocess follows the precedent tests/test_paid_path.py set for tools/paid_path_gate.py, and tests the same artifact check.sh runs, argument parsing included.

I8 — Page citations on both surfaces, and pnk doctor's text yield (20260729 04:55)

HIGH — pinakes_get on a PDF crashed, and no test could have caught it. document() read the source with read_text(encoding="utf-8") inside a try guarded by except OSError. UnicodeDecodeError is a ValueError, so the guard never applied and the traceback escaped through the MCP surface. It survived since v0.1 because no test ever called pinakes_get on a PDF — the serve suite's KB is two markdown files, and every PDF test lives in a module that never builds a server. A gap between two test modules is invisible to both.

HIGH — the plan's page_start == p assertion is wrong, and would have failed on correct code. The I8 draft specifies that every chunk covering the traced offset "reports page_start == p". A chunk that straddles a page break starts on the earlier page, so a word on the later one legitimately sits inside a chunk whose page_start is smaller — which I5 explicitly allows and which the citation renders as p1-2. The trace asserts page_start <= p <= page_end instead. The draft had already corrected "exactly one chunk" to "at least one" for the same fixture; the page assertion needed the same correction and did not get it.

MEDIUM — the stale_extraction row understated its own gap by half. DESIGN §4.7's pending amendment said the marker "today reaches the CLI's Passage but stops there", so I8 would carry it to the agent surface. It reached the CLI's Passage object and was then dropped by the CLI renderer too — computed in search.py, surfaced nowhere. A field that exists in a dataclass reads, at review time, like a field that is displayed.

MEDIUM — the free per-page yield lived inside the only module allowed to import anthropic. survey_free_yield measures what pypdfium2 got out of a page; nothing about it is paid. But it sat in extract/claude.py, so pnk doctor — a free command — could not consume it without importing the paid path to ask a free question, against CLAUDE.md's own "never probe a backend by loading it". Moved to extract/pageyield.py. The alternative, a second per-page loop in doctor.py, would have been a second definition of a measurement that decides whether to spend money.

A dead statistic in a shipped template. The notes template's [budget] comment told every new KB that "no shipped code path spends money" — written when that was true, still shipping three releases later. docs/GUIDE.md said the paid extractor was "built but in no release yet". Both are the same failure as the four README claims found at 0.1.2: prose drifts toward the design, because the design is what you are thinking about while writing it. Neither was in the increment's scope; both were found by reading the files the increment touched for other reasons.

Mutation testing found the test whose name was stronger than its assertion. Twelve of thirteen mutations were detected. The survivor deleted pnk doctor's unmeasured-document tally, and test_a_swept_cache_entry_is_counted_as_unmeasured_rather_than_as_a_pass stayed green — because that test sweeps the whole cache and reads a branch that counts documents rather than the tally. The mixed case, where some documents measure and others do not, is the one the tally exists for, and it had no test. Its name claimed the general property; its body tested the degenerate one.

The review pass over I8's own diff

Three defects, all in pnk doctor's new check, all found by reading it adversarially rather than by any test:

HIGH — the health check crashed on an unhealthy KB. is_paid_backend raises BackendUnknownError on a name it does not recognise, and the check passed it every PDF's recorded backend. A KB indexed by a newer Pinakes, or with an extra since uninstalled, would make pnk doctor itself raise — the one command someone runs because their KB is in a state they do not understand. §4.4's coherence check has carried the identical guard, with the identical comment, since I5; the new code was written beside it and did not copy it.

MEDIUM — a KB whose PDFs are all paid-extracted got a permanent, unclearable warning, with a remedy (pnk sync) that on those documents spends money. The check deliberately skips paid-extracted documents, then reported the resulting empty measurement through the branch meant for a swept cache. Skipped-on-purpose and lost look identical to a counter.

LOW — a single out-of-range page bound was reported as a backwards range. page_start=5 on a two-page document read "pages 5-2 is not a range within it", because the bounds were validated after the omitted one was defaulted. It describes a range the caller never asked for, and reads as Pinakes' mistake rather than a bad argument. Found by running the tool, not by reading it.

What the tests could not have caught. All three needed either a KB state no fixture builds (an unknown backend name, wholly paid extraction) or a human reading an error message. The increment's own tests were green throughout, and so was a sixteen-mutation pass — mutation only perturbs cases somebody already thought of, which is the same limit that let the fragment tooling ship a duplicate-heading bug at the 0.3.0 release.

I9 — Auditing the verification table (20260729 05:40)

HIGH — the table that verifies everything verified nothing. plans/20260727_1543-v0.2.md ends with 98 rows, each promising a property and naming the test that holds it, under a preamble reading "a promise in a section with no owner is a wish". 61 of the 98 test paths did not resolve. Not because the properties went untested — nearly all are tested, usually under a better name than the plan guessed — but because the paths were written before the tests existed and implementation renamed them.

The failure is not the renaming. It is that nothing ever read the table, so it could drift a row at a time for nine increments with every gate green. A table of test paths is prose until something executes it, and prose about tests reads exactly like tests. The fix is not a better table: it is tests/test_verification.py, which resolves every reference in docs/VERIFICATION.md and fails on the first one that does not exist. The document can now go stale exactly once — in the commit that breaks it.

The audit found a real gap on its first run, which is the argument for doing it. test_every_v02_check_appears was assigned to I8, named in the table, and never written. Writing it (as test_every_doctor_check_is_exercised_by_a_test) immediately found five pnk doctor checks with no test at all: template, reranker, model cache, extensions, links. Link coverage is a §6.2 promise; the reranker check exists so a health check does not download weights. Both had shipped untested since I11.

MEDIUM — I wrote the exact CI assertion the plan warned against, and only running it caught it. The plan says the core-only wheel smoke must use "a core-only KB that does not need embeddings, because today's smoke KB fails on sentence-transformers long before it reaches an extractor, so the assertion would prove nothing". I built a PDF-only KB believing that satisfied it, and it does not: the embedding backend loads before any extractor, so pnk sync on a PDF-only core install still fails on pinakes[st]. My grep -q 'pinakes\[' passed — against the wrong extra. pnk doctor is the only surface that reaches the extractor question on a core-only install, because it reports a failing backend as a check and carries on. Reading the plan's warning was not enough to avoid the thing the plan warned about; running the command was.

A plan is a historical record, and correcting it would have destroyed the evidence. The temptation was to fix the 61 paths in place. That would have erased the only proof that predicted test names drift — and with it the reason tests/test_verification.py needs to exist. The plan keeps its predictions under a dated supersession note; the resolved mapping lives in docs/.

A sidecar that would not parse was replaced by a freshly minted one (20260729 07:26)

HIGH — the one failure the design says is unrecoverable, shipped since v0.1 and live in 0.4.0 on PyPI. walk_sources dropped a sidecar it could not read (except PinakesError: continue, sync.py:385) so that one bad file would not stop the walk. That was right. What it did not account for is that the document then matches DESIGN §6.4's "new path, no sidecar" row — and the mint path wrote a freshly minted sidecar over the file still holding the document's permanent ULID. Every inbound pnk:// link points at the id that was destroyed, and there is no migration machinery by design.

Three things made it invisible:

• pnk sync reported success. report.ok was true, failures empty, 1 indexed.
• pnk doctor afterwards reported sidecars: N readable, duplicate ids: none, failures: none — every check green, because the unparseable file no longer existed. The skip site's own comment said "reported by pnk doctor"; that safety net could never fire, because syncing repairs the symptom by destroying the evidence.
• The module that owns the risk had already named it. sidecar.write's atomic-rename comment calls ULID loss "the one failure in this module that no later command could repair" — and then handed the file to a caller that overwrote it deliberately. A guard written against a torn write says nothing about a deliberate one.

How it was found, and what that says. Not by a test — by hand-authoring L1's partner corpus with one deliberately unresolvable link, syncing it, and noticing that pnk doctor reported 10 links where the density gate had just counted 13. The discrepancy was three links, all on one document, and that document's sidecar had a new ULID and a created stamp from the sync. A second, independent count of the same population is what exposed it; every check that read only the post-sync state agreed with itself. L7 requires the gate's number and doctor's number to be the same population for a different reason — so that a user and CI cannot disagree — and this is the argument for computing both at all.

The fix, and one guard that was removed for failing its own mutation test. Minting goes through a new sidecar.create, which refuses where a file exists; the refusal lives at the write rather than in the caller, because "the only caller that reaches it" is a property of today's code. A matching guard added to the --index-only branch of _mint proved undetectable by mutation — deleting it changed no observable behaviour, only which of two SidecarErrors was reported, because the indexing path re-reads the sidecar for its metadata and that read refuses first. It was removed rather than kept, and _mint's docstring records why, so a later reader does not "restore the missing check". A guard that cannot be mutated is not a guard; keeping it would have been the kind of decoration this project's mutation step exists to catch.

The adversarial pass found the bigger half. The fix as first written covered only the case where the document is absent from the index — a fresh KB, a fresh clone, a --rebuild. For a document already indexed whose content is unchanged, pairing yields RefreshMetadata, and that branch sits outside _apply's per-document try, so _refresh_metadata's re-read of the sidecar raised straight through _apply, the action loop and sync(). One hand-broken file aborted the entire corpus: no failures row, no set_meta, no commit, and every document after it unprocessed — contradicting this module's own opening promise and docs/CLI.md's "failures are recorded, the run continues". That is the likeliest route in: edit a link by hand, re-sync. Three paths existed for one cause (Mint, Reembed, RefreshMetadata) and each behaved differently; they now report identically. The lesson is about where the first fix stopped: it was written against the reproduction, and the reproduction was a fresh KB because that is what a corpus author happens to have. A fix aimed at a repro covers the repro's path.

Two smaller things the same pass caught, both about honesty rather than correctness. The refusal said only "already exists, so a freshly minted sidecar cannot be written over it" — which reads like a Pinakes bug (of course it exists) and says nothing about the character the user mistyped, while DESIGN, the changelog and the commit message all claimed it named the parse error. The walk has to swallow that error to keep walking, so the mint path now re-reads the one file to recover it. And the remedy said "repair the file rather than deleting it — it holds the permanent ULID", which is false for the second shape the tests deliberately parametrise over: id: not-a-ulid has no ULID to repair to, and a user in a blocked pre-commit was being told not to do the only thing that unblocks them.

What the tests are parametrised over, and why. Two unrelated parse failures — a malformed link URI and a malformed id. The defect is any PinakesError from read_sidecar reaching the mint path, and a test written only against a bad link would have gone quiet the moment link parsing moved.

L1 — The partner corpus and the density gate (20260729 08:47)

HIGH — the gate did not gate the one shape it was built for. degrees was keyed by basename (path.name), so two documents sharing a filename in different folders collapsed to one key and the later-sorted one overwrote the earlier. Demonstrated against the shipped gate: a degree-6 hub — 50% above the cap of 4 — behind docs/aaa/policy.md exited 0 and was reported as "worst degree 1 (policy.md)". Density alone permits one hub wired to everything; the degree cap exists separately precisely to catch that, and a basename key is the single way it cannot. Now keyed by path relative to the KB root. The committed corpora are flat, so nothing in this repo would ever have exercised it — the fixture had to be built to find it.

HIGH — the gate counted sidecars where it meant documents, and was wrong in both directions. An orphaned sidecar (which pnk sync deliberately keeps) inflated the denominator: 8 of 10 real documents linked read as 27% and passed a 35% cap. A document whose sidecar had not been minted was invisible, so the gate reported nonsense on any KB where sync had not run. Documents now come from [sources] include, which is what the word means.

HIGH — two documents kept a privacy claim the increment made false. README.md still said "The sole KB here is a small synthetic corpus" and CLAUDE.md "The only KB here is the synthetic demo corpus", while DESIGN.md was updated in the same commit to "the two synthetic corpora". The repo contradicted itself about what had been committed, in the section a reader consults because they are worried about exactly that. The audit-the-neighbourhood rule exists for this and I applied it to DESIGN alone — the file I was already editing — which is the failure mode the rule describes.

A claim that was true by coincidence, stated as if by construction. The gate's docstring, check.sh and the changelog all said it "counts the same population pnk doctor reports". The link counts are the same population, by construction. The document counts are not: doctor counts indexed documents, the gate counts files matching include. They agree on the committed corpora and nothing makes them. The wording now says which half is guaranteed.

Two gates and no test that either still exists. L1 added a check.sh gate and a CI job and asserted neither, so deleting either left the suite green — in a repo that already has test_check_sh_declares_the_pdf_quality_guard, written for that exact failure. The convention was there; I did not apply it. Both are pinned now, and CI's negative step additionally greps for the message rather than accepting any non-zero exit, since a crash, a missing corpus, or uv itself falling over all satisfy the weaker check.

What the corpus taught about relations. counterpart was used both as a reciprocated 1:1 pairing (inward loan ↔ outward loan) and as a loose association (courier requirements → outward loan). A later increment reading counterpart as a pairing would be misinformed by its own fixture. Now governs.

The self-form fixture is not a fixed point of the product's own writer. sidecar.write resolves self to a ULID on write, so anything that reads and rewrites that file destroys the trap L2 needs — and pnk link (L6) writes exactly that key. The test catches it, but a long way from the cause, so the hazard is named in the test rather than left to be rediscovered.

Mutation, twice. Seven targets before the review, seven after the gate was rewritten. Two mutations appeared to survive and were worth more than the ones that failed: the first had not applied at all — a str.replace searching for 'self' where the source says "self", the exact no-op conftest._rewrite refuses, met in my own mutation harness, which now asserts the substitution happened. The second was real: nothing asserted that the report prints the cap in force rather than the module default, so --max-density 0.1 printed "27% of the 35% cap" and then failed the corpus in the next line.

And the verification gate caught its own author. Renaming test_the_committed_split_is_what_pnk_doctor_counts (misnamed — it never consulted pnk doctor) turned tests/test_verification.py red until docs/VERIFICATION.md was updated with it. That is I9 working on the first increment after it shipped.

L2 — Reverse-scan (20260730 16:51)

One root cause behind all three HIGH findings: I bypassed manifest.load and kept none of what it was doing. The bypass is right — a partner may run a newer Pinakes whose manifest mentions keys this one has never heard of, and refusing to read a neighbour's inbound links over that would make every connected KB a version dependency of every other. But load is not only a parser; it is also the place that rejects an absolute [sources] roots, rejects .. in one, and validates the include patterns. Reading the TOML directly removed all of it and replaced none of it, and the partner's manifest is input this KB does not control. Every one of the three failures below is that same sentence.

A partner renaming its own docs/ silently deleted every inbound row it had. A roots entry that is not a directory was a quiet continue, so the walk yielded zero sidecars, reported complete=True, and the caller did exactly what it is written to do with a complete walk: delete and replace. Reproduced — rows 1 → 0, link_scan empty, last_scan stamped fresh, so the retry was suppressed for a full window too. This is precisely the mass deletion the complete flag exists to prevent, arriving through the one door the flag was not watching, and no "successful walk" test could ever have caught it because they all leave the partner's sidecars where they are. A missing root is now a walk failure with a reason.

The partner's exclude was ignored, while a comment claimed otherwise. sidecars_under's docstring said a document "whose document was excluded" contributes nothing; it read only roots and include. The shipped notes template stamps exclude = ["**/drafts/**"], so this is not an exotic configuration — it is the shape of every KB pnk init creates, and the scan was recording inbound links from documents the partner's own KB does not contain.

A partner's manifest could crash pnk sync on a git hook. The sidecars_under call sat outside the try, and Path.glob raises on patterns manifest.load would have rejected — NotImplementedError for a non-relative pattern, ValueError for an empty one. Both escaped sync() entirely. The module's central promise is "nothing here raises", precisely so a partner that is merely broken cannot block a commit; the one call that could raise was the one left outside.

Two tests that could not fail, both of mine. test_the_partner_is_never_locked asserted the partner had no .pinakes/ — on a fixture where the partner was never synced, so the directory had never existed. It proved nothing was created and nothing whatever about locking; it now holds the partner's real SyncLock while the local sync runs. And a test asserting no SQLite connection was left open re-asserted pre-existing sync() behaviour (_run's finally: close() always releases it), so no L2-shaped defect could have made it fail. Deleted rather than kept: a test that cannot fail is worse than no test, because it is counted.

A failed local run blamed the partner. known_documents is read from the index, so a document that failed to index this run is absent from it — and a genuine inbound link was then reported as pointing at a document this KB does not have. It does have it; it failed to index it. The local picture is now passed as None on a failed or budget-stopped run, which suppresses that check without touching the rows, since the rows come from the partner and owe nothing to our state. _run already guards active_content_hashes on report.ok for the same class of reason — the precedent was there.

Dead code that credited itself with someone else's work. ScanResult.delisted and the known_kb_ids parameter were computed every sync, complete with a docstring explaining that the rows "are removed" — by a function that never read either. The sweep is store.forget_reverse_links, which takes the manifest's ids directly. Removed, along with the SELECT DISTINCT that fed it.

Mutation: 11 targets before the review, 5 more after, all detected. The one apparent survivor was equivalent code rather than a gap — taking src_kb_id from the declared id instead of the partner's own is indistinguishable wherever a row is written, because the mismatch guard refuses first. The guard is what carries the weight, so the test asserts what makes the assignment moot: a mismatched id writes no rows and no kb_refs entry.

And a test premise of mine was wrong, which the failure said plainly. _replace_links only runs for a document that gets an action, so the reverse-then-authored ordering needs the document to actually change — a second sync skips everything and rewrites nothing. Worth keeping because it is a fact about when authored links are re-asserted at all, not just about this test.

L3–L4 — The traversal core and pnk links (20260730 18:06)

Four HIGH findings, all in the properties the increment's own prose claimed loudest. That is the pattern worth keeping: the module docstring argued at length for double-capping, precedence and server-side clamping, and each of those three was where the defect was. Writing the argument down appears to have substituted for checking it.

The response was half-capped. max_rows and token_budget gated neighbours; the frontier was appended to unconditionally. Measured: a caller asking for one row received 1,000 frontier entries — and the frontier is the part an agent parses to decide what to ask next. Now capped, and ordered so that entries about nodes you did not get come first: capping without that ordering let the depth notes of accepted nodes fill the whole budget and crowd out every rows note, so a caller asking for 2 of 5 was told nothing about the 3 it missed.

"Every bound is clamped server-side" was true of two of the four. max_rows=10**9 returned 3,660 rows with an empty truncated. Three documents said otherwise. Once this is reachable over MCP the caller supplying max_rows is the untrusted party, so the sentence was not merely inaccurate.

A frontier entry contradicted the answer beside it. A node dropped by fan-out at one hop and reached at another kept its fanout entry — while sitting in neighbours and having been expanded. FrontierEntry's own docstring says "discovered and not expanded". Stale drops are now retracted at return; terminal and depth are kept, because those describe accepted nodes deliberately not expanded, which is the contract rather than a contradiction of it.

Half the stated precedence was inverted. The row and token checks ran before terminality was consulted, so a terminal neighbour dropped by the row cap reported rows — inviting a retry with a smaller request, which cannot help. Of the ten pairs the declared order implies, five were backwards and exactly one was tested: the one the code happened to honour.

The gate had three separate ways of being vacuous, and its docstring was an essay about gates that cannot fail.

• It passed against a traverse() that returned an empty Result — every check was one-sided, so zero neighbours satisfied all of them. Now equalities.
• It imported MAX_DEPTH and MAX_ADJACENT_K from the code it gates and compared them with themselves. Raising the caps to 10 and 150 moved the walk and the gate still passed, while docs/MANIFEST.md went on promising 64. The documented numbers are now literals in the gate — a second copy, which is the only thing that makes a silent change show up.
• It had no negative check, in a repo where the immediately preceding increment added one to its sibling job and a test that guards it. Added, with a --expect-depth override so CI can drive the gate into failure on purpose and assert the stated reason.

Two more the same pass found. The row cap truncated by parent-expansion order while ranking was per-parent, so a top-ranked neighbour behind a low-ranked parent lost to a worthless one in front of it — the same mistake as truncate-then-rank, one level up. And node-level row dedup silently dropped a second distinct relation to the same target, in a module whose contract is that a fact about the graph is returned rather than dropped; rows are now deduped per edge while expansion stays per node.

A dead sort term with a docstring defending it. _rank sorted by (-weight, distance, node_key) and explained that a nearer neighbour of equal weight ranks higher. _rank is called with one hop's candidates, so distance was constant in every sort. Removing it changed nothing — which is how it was found, and is the argument for deleting rather than believing prose.

A test that could not hold its name. test_depth_counts_logical_hops_not_physical_edges had no hub in its fixture and its own docstring conceded the core never sees one; it was a second copy of the clamp test wearing a larger claim, and docs/VERIFICATION.md cited it for a promise it could not carry. Renamed to what it actually checks. The logical-hop promise belongs to the provider that composes hubs.

And new behaviour shipped without tests. [retrieval] adjacent_k and _toml.integer(maximum=) had none — the commit message's claim that a value above the cap is refused rather than clamped was asserted and never executed, against this project's own rule that tests ship in the increment that introduces the behaviour.

A process failure worth recording separately. L4 was built in L3's worktree while L3's adversarial review was still reading it, so the reviewer found the tree dirty with a parallel increment's work and had to run every probe against a copy. It cost the review nothing this time because L4 added files rather than editing traverse.py, but that was luck. One increment, one worktree, and the review finishes before the next one starts.

Four silent str.replace no-ops this session, one of which spliced a new import into the middle of an existing one and produced a nonsense symbol. It is the same failure conftest._rewrite exists to refuse, met in editing rather than in a fixture. Non-trivial edits now go through a tool that errors when its anchor does not match.

L5 — pinakes_links on the MCP surface (20260731 11:29)

The defect was in the field nobody thought to assert

L5's own mutation pass killed all three of the targets the plan named. An adversarial review then mutated eight more payload fields and watched every one survive the full 887-test suite. Two of those were real defects, not merely untested:

• direction was keyed by node, while a row is (node, rel). Given a --related--> b and b --cites--> a, asking about a reported the citation as running from a — the opposite of what someone wrote. Shipped in L4, copied verbatim into L5, wrong on both surfaces. The provider's own docstring argued the case for the key it used: "Keyed by node, because a node reached both ways is still one neighbour" — true of the node, irrelevant to the row.
• DIRECTIONS was defined and never enforced. edges_of tests in ("out", "both") and in ("in", "both"), so direction="outbound" ran neither query and returned a confident empty answer with a "no links from here" hint. argparse choices covered the CLI; the MCP surface, the one an untrusted model types into, had nothing.

The lesson that generalises: a field with no assertion is a field that can be a constant. Ask of each one, "which mutation would this catch?" — not "is it correct?". scored_by_query, the field L3's docstring calls load-bearing, could be frozen to True; unresolved, whose contract says "returned, never dropped", could be frozen to [].

A tidy fixture defeats a mutation test. Three fields survived even after tests were written for them, because the KB-backed fixtures were too clean: the fake embedding backend's vectors are orthonormal, so every cosine is exactly 1.0 and deleting round() changes nothing; nothing hit a response cap, so truncated could be frozen empty; no frontier entry sat past distance 1. The fix was to build the dataclasses directly and take the fixture out of the question.

Two copies of one payload had already drifted — the MCP frontier carried a distance the CLI's did not, scored_by_query reached only one of them, unresolved dropped a kb_id its sibling lists carried. Neither failed, because nothing compared them. They now share pinakes.graph.present, and a test asserts both surfaces project the same keys.

Calling a tool is not the same as exercising it. The free-path gate was strengthened to invoke pinakes_links rather than only list it — but the fixture KB had one document and no links, so the whole neighbour projection never executed. A raise SystemExit planted in that loop never fired. The fixture now authors one intra-KB and one unreachable-KB link, and the same probe fires.

The fix for a wrong answer produced a differently wrong answer, and the tests written with it could not see that either. Keying directions by (node, rel) was right; merging to both across expansions was not. directions accumulates over the whole walk, so an edge discovered while expanding an unrelated parent rewrote a row already emitted from the start — and a row's direction then changed with --depth, to exactly the untruth the fix was written to remove. Both new direction tests ran at depth=1, where the start is the only parent, so neither could reach it. A second adversarial pass found it by varying the one parameter the tests held fixed.

The generalisation: when a fix adds a rule, test the axis the rule is defined over. The rule was about which expansion a direction came from, and every test pinned a single expansion.

A third pass found no new defect in the traversal itself, and four in what surrounded it. The (node, rel) scheme was probed against a reciprocal pair, a mutual same-rel pair, each direction, a self-loop, a 3-cycle, a node reached at two different hops by different relations, a node reached by two parents in one hop with opposite directions, and a node dropped by fan-out then re-reached — all correct. What was still wrong sat one layer out: an assignment nobody asserted, a message worded from the wrong end, a branch ordered ahead of a better one, and an assertion satisfied by a substring.

Two of those are worth naming as patterns:

• assert "-> related: b" in output passed on <-> related: b. A substring assertion over rendered text will match a longer glyph containing the shorter one, so dropping the outbound arrow entirely left the test green. Match whole lines when asserting on human output.
• Splitting f(x, scores=s) into f(x); f.scores = s moved a value out of the type checker's reach. The construction was covered by the tests that built providers directly; the assignment was covered by nothing, and deleting it disabled query ranking with every gate green. When a refactor turns an argument into a mutation, the mutation needs its own assertion at its own call site — and there were two call sites.

Left for the graph release (L3 core, predating this increment, found while probing): a node dropped by fan-out at hop 1 and re-reached at hop 2 is emitted with distance: 2 although it is one authored hop from the start; and a self-loop (a --sameas--> a) is dropped entirely — not a neighbour, not unresolved, not on the frontier.

A fix applied to one surface is half a fix. Round 3 gave pinakes_links the rule that a narrowed walk reports the narrowing before it reports dangling links — and left pnk links branching on unresolved alone, in the same commit, so the CLI told a user their links "resolve to nothing" about a document with a live neighbour one dropped --rel away. Both the docs and the changelog described the MCP behaviour as though it were both. The two surfaces now share present.is_filtered and present.arrow, which is the only way this stops recurring: the rule has to live in one place, not be applied twice.

A remedy in an error message is a claim, and it was false. The dangling-links hint sent the caller to pnk doctor — but doctor._links inspects only the destination side of local sidecar rows, so when the missing endpoint is the link's source (a deleted document whose outbound rows survive the soft delete) doctor reports links: OK and contradicts the message that sent you there. Dropped the clause; extending that check belongs to L7, which owns doctor's link coverage.

Four review rounds, each finding real defects in the previous round's fix, then converging: 11 findings, then 11 with one HIGH, then 7 with none, then 5 with none. What the last two rounds found was never the traversal — it was the layer around it: an assignment nobody asserted, an assertion satisfied by a substring, a message worded from the wrong end, a branch ordered ahead of a better one, and a rule applied to one of two surfaces.

The rule two rounds were spent getting right had no test that could detect its inversion. Round 5 found the shipped behaviour correct on both surfaces and the precedence — filter before dangling before "no links" — freely reversible with the suite green. The cause was a fixture that could not make both conditions true at once: --rel narrows provider.unresolved as well as the neighbours (edges_of receives the same rel), so a rel-filtered call leaves unresolved empty and the branch being out-ranked never competes. --direction is the lever that does it — an outbound link that dangles and an inbound one that is live. The assertion that named the defect in its own message ("one dropped argument away from a live neighbour") was the vacuous one.

Test the discriminating case, not the two sides separately. A precedence rule is only observable where both branches are eligible; a fixture that satisfies one at a time asserts the wording of each and the order of neither.

L5b — ruamel.yaml replaces pyyaml in the sidecar (20260731 11:29)

Swapping a YAML library is not a swap

The plan predicted three failures precisely, and all three landed as written — which is worth recording because it is the first increment in this project where that happened. It named the 872nd test ({id: x, : }, which ruamel parses, so the case fell through to the id check and the parse-error branch had been asserting nothing), the free-path gate being red on day one, and the ScalarBoolean coercion being insufficient at one level.

The free-path gate was defeated by its own harness, and I wrote the defect. _author_links in tests/free_path_run.py — added in L5 to close a coverage hole — wrote a sidecar through yaml.safe_dump. That put yaml into the very module list the gate inspects, and it was also the last PyYAML sidecar writer in the repo: a fixture written by one library and read by another, which is exactly the divergence the gate exists to forbid.

existing[:] = keep wipes ruamel's comment metadata outright. Reconciling a sequence by rebuilding a keep-list destroys CommentedSeq.ca.items entirely — every comment in the block, not just the removed entry's — while del existing[index] shifts the survivors. Measured: {} against {0: '# first', 1: '# third'}. Both merge functions had it.

A comment before a sequence entry belongs to the entry above it, exactly as it does for a mapping key. The plan pins the deletion limitation for mapping keys; it is broader than that. After deleting the middle of three commented links, # first stays correct, # second reattaches to the third link, and # third disappears. The surviving links are all correct; the prose beside one of them is not.

"Unobservable" was the wrong conclusion; "observable only where something else is broken" was the right one. The plan's "assign a known key only when its value actually changed" looked untestable — every known-key value is the node read out of the document and written straight back, so a write of an unchanged document is already a no-op. Two attempts at a test passed against the mutated source and I wrote that no mutation of it could fail. A reviewer then removed the rule and the committed corpora stopped round-tripping: the short-circuit was masking the duplicate-link defect below, not proving its own redundancy. Once that was fixed the rule really was unobservable — but the claim was true by accident for two commits, and the difference is exactly what an adversarial pass is for.

-x makes a mutation look like it was caught by the wrong test. Two links mutations appeared to be killed only by an unrelated pre-existing test; without -x both were also killed by the test written for them. Run the mutation pass without early exit, or the report is about test ordering.

A merge key must be the identity the storage layer already uses. Reconciling links on to alone looked sufficient and is undefined the moment two links point at one document with different relations — which _links() accepts and the index stores as two rows, its primary key including rel. Measured on the version that had it: dropping an unrelated third link rewrote the first link's rel to the second's and deleted the second, leaving one row carrying the wrong relation under the other's comment. The index's own PRIMARY KEY was the answer, and it was already written down in store.py.

A recursive rule needs a depth bound as much as a base case. "A key absent from the new mapping is deleted" is required at the top of provenance — or --force leaves a false paid claim behind — and destructive one level down, where with_extraction_provenance builds a plain four-key replacement and the user's own reviewed_by sits beside content_hash. One sentence, two opposite correct answers, distinguished only by depth.

The exit criterion was the thing nobody ran. The plan's one falsifiable sentence — every committed sidecar still round-trips — had no test, and running it by hand found a pnk://self/… entry in partner-kb being deleted, rebuilt without its unknown per-link keys, and moved to the end of its block. _links() expands self on read, so the loaded entry's raw to never equals anything in the reconciliation set. The docs bounded the invariant with "pnk://self/… expansion", which reads as the URI text changes and not the entry is rebuilt — a documented exclusion that quietly covered a defect.

Quoting was applied on the path that was tested and not on the path that ships. Decision 23's predicate reached the merge branch and the mint, but not the branch taken when a key first appears — which is the branch pnk link will follow on a sidecar that has no links: yet, i.e. almost all of them. Three quoting mutations survived the whole suite.

An error message is part of the interface. Three of this increment's breaking changes surfaced as TaggedScalar, ScalarFloat and OctalInt — ruamel class names, from a library the user never chose, with no remedy. The type is not what they need; "quote it, or drop the tag" is.

A fix instruction can carry its own defects, and two of pass 6's did. Keying links on the (to, rel) pair made the pair the entire content of an entry, so no matched entry was ever updated and every rel edit became a delete plus an append — landing straight in the comment misattribution the rule was written to avoid. And "positional fallback among equal pairs" was too vague to implement; what I wrote from it used a set, which collapses two identical entries: three links in, one out. The final rule needed three explicit clauses — resolve before comparing, multiplicity never a set, assign rel in place — each naming the shipped version that got it wrong.

"Exactly the call being protected" was true of the call and false of the argument. The JSON-encodability check ran json.dumps(extra, sort_keys=True, ensure_ascii=False) — the same function store.dumps_metadata calls — over extra alone. _metadata() hands that function {"tags": …, "provenance": …, **extra}. A uniformly int-keyed {1: a} sorts perfectly on its own and becomes mixed the moment the string keys join it, so pnk sync still crashed. Checking the parts is not checking the whole, and the docstring asserting otherwise is what made it look done.

Two tests could not observe what they claimed, for the same reason. A plain read-write of an unchanged document short-circuits before the merge runs, so a wanted that deduplicates survived test_two_identical_link_entries_both_survive and a set-based collapse was invisible. Any test of reconciliation has to change something first, or it is testing the short-circuit.

A warning is not an error, and a library that downgrades one is changing behaviour. A reused anchor name raised ComposerError — a YAMLError — before the swap, and after it the document loads, every alias resolves to the last anchor of that name, and the only signal is a ReusedAnchorWarning on stderr. Three consequences, none visible in a passing suite: the value silently changes; read()'s except YAMLError never sees it, because a Warning is not one; and under this project's filterwarnings = ["error"] it escapes as a bare warning traceback rather than a named error. Promoting it at the load makes the outcome independent of whatever warning filters the calling program happens to have set — which is the right place for a property of the file format to live.

An exclusion list is a set of claims, and claims rot. Every bound on the byte-identity invariant — indentation, !! tags, anchors, CRLF, BOM, document markers — was prose in a table until it was pinned by a test. Writing those tests measured two behaviours that were not on the list at all: a plain (non-recursive) anchor on an empty value is destroyed, where the list named only the self-referential case; and a file with no trailing newline gains one. Both are byte changes to a file nobody edited, which is exactly what the invariant claims does not happen. A bound stated only in prose cannot notice the library moving under it, and cannot be wrong out loud.

It also falsified a changelog line I had written: "!!int, !!float, !!bool, !!seq and !!map keep working — verified". Verified of loading; the tag itself is dropped on write, so !!int 3 comes back as 3. True of the value, false of the invariant, and the word "verified" is what made it read as covering both.

A gate that has never been shown to fail is a claim too. The AST scan now proves it sees all four shapes of a planted import — including the function-scoped one an import walk cannot reach — and does not fire on any of the four legal ruamel forms, every one of which contains "yaml". The stub signature test proves it can fail: writing transform into the expected set failed it, because transform belongs to dump rather than __init__.

"PyYAML left the runtime" is true of what Pinakes declares and false of what a user's machine has. Measured on a built wheel: bare, yaml is absent; pinakes[light] has it, transitively from huggingface_hub. starlette and uvicorn list it too, but only under an extra they do not pull. So the CI assertion is correctly scoped to the bare wheel — Pinakes never asks for PyYAML — and the consequence is the part worth remembering: import yaml will succeed in a real install, so a stray import in src/ would quietly work instead of failing loudly. That is what makes the AST scan load-bearing rather than a second belt.

The worst defect in this increment was a rule the plan itself wrote. "One instance, reused rather than reconstructed per call", justified by 282 µs against 399 µs. ruamel keeps the %YAML directive from the last load() on the instance and applies it to every later load and dump: read a sidecar carrying %YAML 1.1, then write an unrelated one that never did, and it comes back with the directive injected and country: NO rewritten to false. The exact corruption this increment exists to remove, reintroduced across documents, in exchange for 117 microseconds — and freshly minted sidecars are contaminated the same way. Nothing softer fixes it: resetting version after the load still emits the directive, pinning it up front is overwritten by the next load. A performance justification measured in microseconds should be read as an argument that the optimisation does not matter.

A gate that never reads the artifact it guards is checking a copy. The stub-signature test listed the symbols in a hand-written Python dict, checked them with hasattr, and compared against hardcoded signature supersets — so a stub declaring a parameter ruamel does not have was green under pytest and pyright, which is the single failure decision 20 exists to catch. It parses the .pyi files with ast now.

A fixture can be right for the wrong reason and hide the defect it was written for. The two-links-sharing-a-to test edited the first entry, and entries are walked in descending index order — so the single-pass form it was meant to catch happened to produce the correct answer. Editing the second entry is the discriminating case: its fallback claims the link the first was owed exactly, and both relations end up swapped under the wrong comments. Two of this increment's tests have now needed the specific case rather than a representative one.

main was red for four merges, and local check.sh could not have known (20260801 06:05)

Four tests written across L6 and L7 passed on macOS and failed on CI, so 2314dea (L6) and ed01b00 merged onto a red main and stayed there until L8's verification step 1 looked.

Two causes, one shape: a test that cannot build its precondition does not skip — it asserts the wrong thing.

• chmod(0o000) is not a portable way to deny a read. Three fixtures built an "unreadable directory" that CI read anyway: pnk link reported no pinakes.toml there where the test demanded Permission denied, and 'docs/locked/x.md' is not a document in this KB where it demanded cannot be read.

The first fix was wrong, and its wrongness is the lesson. It probed whether permissions are enforced and skipped when they are not — reasoning that CI runs as root. CI is not root: the probe reported permissions enforced, did not skip, and the run failed identically. Whatever that runner does with a mode-000 directory, is_file() neither succeeded nor raised EACCES.

Skipping was the wrong shape regardless. It disables the guard exactly where it broke — the environment the test could not model is the one that most needed testing. The refusal is now injected: Path.is_file raises PermissionError, which is precisely what the guard exists to catch, on every platform and with no filesystem semantics in the way. A test for "an OSError becomes a PinakesError" should raise an OSError, not arrange for the operating system to. - Two more of the same shape, found only by pushing the fix and watching CI again. A 300-character filename asserted to produce ENAMETOOLONG — the length at which a filesystem says that is a property of the filesystem, and on CI the name was simply not a document. And an embedded NUL in an include pattern asserted to raise from resolve() — on CI it raised nothing, so the test asserted a problem that never occurred. Both now raise the error directly.

Every one of these tests asked the operating system to produce an error, and then asserted on the answer. That is a test of the platform, not of the guard. The guard's contract is "an OSError/ValueError from this call becomes a PinakesError" — so the test should raise one.

• pathlib's wording is not a contract. A test asserted Unacceptable pattern: PosixPath('.'), which CPython renders as Unacceptable pattern: '' on other versions. The increment's promise is that the pattern the author wrote is named and the other include entries survive; that is what it asserts now. Third instance in two increments of asserting a phrase where the property was meant.

The process failure is the larger one. ./check.sh was green before each merge, and green on one developer machine is not green on the three-leg CI matrix — different OS, different Python patch, different privileges. The project rule already says to check whether the latest run on the default branch actually succeeded; it was not checked after either merge, and the second merge landed on top of the first failure without noticing it.

gh run list --branch main belongs in the merge sequence, after the push, not in the next increment's verification step. A red default branch blocks the release either way — finding it two merges later only makes the bisect longer.

G1 — Is the eval reproducible? (20260801 00:52)

Decision 15 said measure before fixing, and the measurement paid for itself twice: once by finding a real defect, and once by contradicting the fix that was nearly written for it.

HIGH — the eval was reproducible by luck, and the luck was invisible. Running the golden set, editing a document, re-syncing, rebuilding and comparing per-question outcomes: the real [light] models agreed everywhere. A low-dimensional fake disagreed on one question in 41 between an incremental sync and a --rebuild. Both facts are the same fact — 384-dimensional cosines almost never tie exactly, and every tiebreak underneath resolved to chunks.id, the rowid, which store.py's own schema comment says has no identity across rebuilds. A property that holds because the corpus never exercises it is not a property, and G5's sign test was about to be built on it. The fix is total ordering on (documents.path, chunks.ordinal) at three sites plus a stable argsort; no measured number moved, which is the right outcome for a change that only breaks ties, and is why this increment rewrites no baseline and needs no amendment to L8 step 5.

HIGH — the fixture was the algorithm, in the one place that judges the algorithm. The first version of the tests used eight dimensions, reasoning "fewer dimensions, more ties". Swept against the genuine pre-fix code, eight and sixteen dimensions both reported zero differences across all four perturbations: collapse the space far enough and every candidate ties, so the ordering underneath stops reaching the top-k at all. The relationship is not monotonic — 32 caught two perturbations, 64 caught one, 128 caught two. Had the sweep not been run, G1 would have shipped a green gate, a green test suite and a live defect, all three agreeing. The sweep is recorded in the gate's own DIM docstring rather than the conclusion alone, because the next person's intuition will be the same as this one's.

HIGH — the mutation harness deleted the thing it was measuring. The first mutation run restored each mutation with git checkout -- <file>. The fix was uncommitted, so the first restore reverted it; every later mutation then failed to apply to code that no longer had the target, and the suite ran green against original code four times while reporting "0 failures" as though the mutations had been survived. It read exactly like a well-tested change. Two rules fall out. Restore from a copy of the mutated-from state, never from HEAD — git checkout restores to the last commit, which is a different thing from "undo my mutation" whenever the work is uncommitted. And a mutation harness must assert that its mutation applied: the rewritten one fails loudly if the target string is not found exactly once, which is what turned three silent no-ops into an error.

MEDIUM — a stable sort needed its own test, and the obvious one was vacuous. kind="stable" changes nothing a repeated run can observe: on a fixed input array NumPy's introsort is deterministic. It changes what happens when the array grows, because partitioning depends on the whole array — measured at 500 of 500 random tie-heavy arrays reordering their original entries. The first test written for it used four tied chunks and passed under the mutation, because NumPy uses an insertion sort below roughly sixteen elements and insertion sort is stable whatever kind says. A fixture can be too small to contain the behaviour it is named after.

What the increment ended up asserting, and at which level. Three end-to-end tests state the property G5 needs, over the committed corpus and questions; four site tests each drive one ordering decision directly and are the mutation targets — one mutation, one failing test, verified for all four. The two levels are not redundant: the end-to-end tests can only observe ties the corpus happens to contain, which is precisely how the defect survived three releases.

The adversarial pass over G1's own diff (20260801 01:25)

Six findings, four of them real defects in work that was already green.

HIGH — a gate advertised a field it had retired. _plant rewrote the reranker's model name and left [retrieval.confidence] fitted_for naming the real one, so _confidence short-circuited and all 41 questions scored unknown. Both the gate's docstring and the tests claimed to compare the confidence label. Naming the reranker was not enough either: the committed thresholds were fitted on a real cross-encoder's logits and sit below every score the fake can emit, so the label became a constant high — still unable to move. Thresholds inside the fake's range give 35 medium / 5 high / 1 low, and the field is finally live. The class of defect matters more than the instance: a fixture that rewires half of a calibrated pair silently disables the thing it was calibrated for, and nothing fails.

HIGH — the plan still asserted what the measurement disproved. Decision 15 says a final tiebreak would be "a provable no-op" because cross-document ties are totalised by documents.path and rowid order is ordinal order. Both premises are true about writes and irrelevant to the output: documents.path cannot separate two chunks of the same document, and an incremental sync by definition does not rewrite the files it did not touch, so rowid order stops matching corpus order at the first re-chunked file. The plan is an executor doc; leaving that cell intact would have licensed a G2–G5 executor to skip a tiebreak for a reason this increment measured to be false.

MEDIUM — half the gate's sweep has never observed anything. Of its four perturbations, added and removed report zero differences against the genuine pre-fix code at every width swept (8/16/32/64/128), while edited and renamed bite. --inject-difference cannot reveal this: it corrupts all four alike. The gate now states it. A gate's own justification is a claim like any other — this one said "it sweeps four ways where the tests exercise one", and two of the four were along for the ride.

MEDIUM — the contract's file table was checked against the wrong question. It compared the two tracks' owned files and never asked what a new gate touches. Every gate edits check.sh, ci.yml and tests/test_check_script.py, which both tracks append to at the end of the same regions; and G1 necessarily edits search.py and store.py, which the table lists under neither track, because reproducibility is a property of core retrieval. Widened, with the reason.

LOW, and recorded rather than fixed — making the BM25 cut total costs a join: +11.5 ms on a 50k-chunk corpus where every chunk matches every term. That is the worst case a planner can be given, the correctness is not optional, and the number now sits in docs/STATUS.md so a later change can argue with it.

What the pass confirmed, having tried to break it: bm25() still resolves with the alias present and returns byte-identical rows; the join multiplies nothing (both sides unique); the load_vectors reordering costs nothing measurable; graph/provider.py, the other caller, reduces to a per-document max and is order-independent; the four site tests each fail against pre-fix code; and the artifact paths, cache keys and macOS wheels in the cross-machine job all resolve.

L6 — pnk link (20260801 01:41)

Every review commit on this increment found defects in the one before it, and most of them found the previous commit's own fix or claim rather than something it had missed — 3ce150e (review 1's containment fix had traded one defect for two), 986faf3 (review 2's fix was right; its stated justification was not), 7b3f0a3 (the escaping-error class sat one line above the try added for it), 9c8f667 (the totality fix re-anchored the walk on the working directory), cdee8d8 (the test for an untested branch entered it, but its assertion held either way), dbebd8b (a severity asserted, not measured), and the last three, which took four goes at one containment rule.

No total is given, deliberately. Three drafts stated one and all three were wrong, because it changes depending on whether 8b and 9b count as rounds of their own — and the last wrong figure was introduced by the commit correcting the one before it. git log main..HEAD is the answer, and it cannot go stale. What follows is the state after all of them, not a log: the rule is to rewrite to the current state rather than layer corrections, and earlier drafts of this fragment broke it four times — describing a concurrency scenario a later round had disproved, calling every self-link a typo after the fix for the other case existed, counting the rounds that had happened when it was written rather than the ones that had, and asserting a safety property ("Path.resolve() is safe at both call sites") that was wrong twice over: strict=False suppresses OSError, not the ValueError an embedded NUL raises, and there are six Path.resolve() sites across the two modules rather than two.

One defect class, six instances, and why fixing it at the call site produced them

HIGH. cli.main catches PinakesError. Anything else is a traceback on a user's terminal — or on an unattended post-commit hook. Six calls in this increment's blast radius raised something else:

1. Path("~nosuchuser/x.md").expanduser() raises RuntimeError, on <source> in link.py. It bought nothing either: a ~ that does expand lands in $HOME and is refused by the containment check on the next line. Copying a call across a boundary copies its justification too, and linkscan's need for it (a [[links.kb]] path may be ~/kbs/partner) did not survive the trip.
2. Path.is_file() ignores ENOENT, ENOTDIR, EBADF and ELOOP and raises everything else — so an unreadable parent directory (EACCES) and an over-long name (ENAMETOOLONG) on the same source path.
3. The is_file()/is_dir() pair one branch over, in _via_alias: a partner KB directory this user cannot read raised PermissionError.
4. resolve_path, on the line immediately above the try just added for (3).
5. The same three, in the module link.py calls into. linkscan.scan_one's docstring promises "Never raises: every failure comes back in issues", and all of them sat in the three lines that ran before any handling did — so pnk sync on a hook became a traceback. There since L2.
6. resolve_path again, bare in scan()'s freshness branch — which plain pnk sync takes, so a partner path that stopped resolving crashed every git commit inside the TTL. The branch had no test at all.

Fixes 1–5 each wrapped the instance in front of them and stopped. What closed the class was moving the guarantee into resolve_path itself — a guarantee three call sites each have to remember is a function with the wrong contract — which then removed the wrappers fixes 4 and 5 had added.

The first version of that fix introduced a worse defect than the one it closed, and this is the part worth keeping. resolve_path was made total: on text no filesystem call accepts it returned Path(raw), the declared text, so an error could still name what the author wrote. That value is relative, and five consumers use it as a filesystem base — (path / MANIFEST_NAME).is_file(), why_not_a_kb, partner_sources, sidecars_under, _doc_id_of. So the walk silently re-anchored on the process's working directory: the precise thing resolve_path's own first paragraph says it exists to prevent, reintroduced four paragraphs below by the round that wrote it.

With a directory of that literal name in the CWD holding a readable pinakes.toml, pnk sync walked the decoy, found nothing, stamped the scan complete — and replace_reverse_links deleted every inbound row the real partner had, with report.ok true and no issue raised. That is the real consequence, and it is silent data loss.

Round 8 also claimed pnk link would write the decoy's ULID into the real sidecar, permanently. It would not, and round 9 reproduced the refusal. _document_in compares an absolute joined.parent.resolve() against the relative root, which can never be is_relative_to, so it fires before any sidecar is read — 'docs/one.md' is outside `partner`, which tells the user the path they typed correctly is wrong and names neither the KB path nor the expansion failure. A message defect, not corruption.

Three things kept that overstatement alive for a round. The true account was already in the tree — link.py's own comment describes the misleading refusal — so the increment carried both versions at once. The regression test's docstring asserted the severe reading, and under the round-7 mutation it failed on its first assertion, so the two that encoded the severe claim were never reached: a test that fails proves the mutation is caught, never that it is caught for the stated reason. And the claim was written from the mechanism (a relative base re-anchors the walk → therefore the walk completes) rather than from running it. That is the same "prose written from the design" failure as the two documentation defects below, in a commit message and a retrospective — the two places where being wrong is hardest to notice later, because nothing executes them.

The answer is None, not a fallback value: text that names no path yields no path, and pyright makes every caller say what it does instead — a type-checked obligation rather than a remembered one, which is the same lesson one level up. The declared text is still what the message names; it was always available as linked.path, which every caller already held. A total function is not automatically a safe one — totality only moves the failure from a raise to a return value, and a return value that is the wrong kind of thing is harder to notice than an exception.

Four tests fail against the round-7 shape, verified by mutation — including the two written for it, though one of those for a different reason than its docstring gave (above).

A defect class is not closed until it has been searched for, and the search is mechanical: list every call in the module that touches the filesystem and ask of each which errno it swallows.

Path.resolve() belongs on that list and was wrongly excused twice. strict=False suppresses OSError; it does not suppress the ValueError raised for an embedded NUL, which tomllib accepts in a manifest and pathlib will not open. Enumerated rather than excused, there are six sites: _document_in (link.py:298) resolves a path built from user text and is now guarded and tested; resolve_path (linkscan.py:178) is the fix above; and sidecars_under has four — anchor, the roots entry, the pattern probe and the per-candidate check — all inside the caller's except (OSError, ValueError, NotImplementedError, PinakesError).

The enumeration is the point: "safe at both call sites" named neither the number nor the reason, so it could not be checked without redoing the work — whereas a count with line numbers is wrong the moment it drifts, and says so. It has drifted twice already: round 8 corrected an earlier version that called two of the sidecars_under sites partner-controlled when one is not, round 10's own fix added the fifth site, making "four" stale in the same commit that relied on it; and round 13's added the sixth the same way.

The containment check took three spellings, and the first two were each wrong in one direction

HIGH. _document_in decides whether a path names a document in this KB.

• joined.resolve() — the original — follows the final symlink before checking, so a symlinked document was refused as "outside this KB", with a remedy repeating the path the user had typed correctly. pnk sync indexes such a file, pnk doctor calls its sidecar readable and pnk links traverses it; only pnk link said it was not there, and nothing could link it in either direction.
• os.path.normpath — round 1's fix — follows nothing, so a symlinked directory under docs/ passed containment: the write went out of the KB through it, and in the other direction minted a permanent pnk:// to a ULID this KB will never index, because Path.glob does not recurse a symlinked directory. It simultaneously refused a legitimate absolute path whose ancestor is a symlink — the ordinary shape on macOS (/tmp → /private/tmp) and behind any symlinked checkout — because manifest.load resolves the root, so a verbatim comparison could never match it.
• joined.parent.resolve() / joined.name is right in both directions. The directory chain is followed, so an escape through it is caught and a symlinked ancestor lands inside; the final component is left alone, so the document's own symlink is irrelevant — which is correct, because Path.glob does yield a symlinked file.

normpath must also not run first: it collapses docs/link-to-elsewhere/../x.md to docs/x.md textually, turning an escaping path into one that looks contained. resolve() on the parent collapses .. after following the links it sits behind.

The docstring was wrong for longer than the code. Two drafts justified the check with "what decides membership is the path under [sources]" — a rule the check has never implemented, since it compares against the KB root. Round 2 quoted that sentence as the lesson and left it in place; round 3 found it still there. The residual it was papering over is now stated instead: a document inside the root but outside [sources] can be linked, and the link will not resolve until that document is ingested. Answering the [sources] question properly means re-implementing walk_sources including its globs, and refusing a "link it now, ingest it next" order of work that costs nothing.

Two documentation claims the code contradicted, in prose written from the design

HIGH. The new pnk link section told the reader that a pnk:// URI pointing at a KB not on this machine is fine because "pnk doctor reports a dangling target; pnk links lists it under unresolved". Neither happens. doctor.py filters its dangling list to this KB — the cross-KB check is L7, the next increment — and provider.py's unresolved carries a docstring explicitly refusing to widen: "a cross-KB target cannot be checked from here without the other KB, and reporting one as unresolved on that basis would be asserting something this index has no standing to know." A reassurance was invented for the one case the section was telling the reader not to worry about, and half of it described something the design had already declined to build.

The replacement prose then made the same mistake twice more, which is the finding worth keeping. Round 1's fix illustrated the missing lock with a post-commit hook firing a paid extraction — a scenario hooks.py structurally prevents. Round 2's fix replaced that with "the one sync that rewrites an existing sidecar is a paid extraction", which sync.py's --force-plus-free---extract override falsifies, and which this increment's own edits to DESIGN, MANIFEST and CLAUDE.md all name the carve-out for. A correction is a diff and earns the same verification as the line it replaces; three rounds of unverified prose about the same paragraph is what happens otherwise.

Fixtures that were representative rather than discriminating

MEDIUM, four times. A test can be green because the code is right or because the input never reaches it, and the two look identical from the outside.

• test_no_line_outside_the_links_block_changes_when_a_link_is_added used a sidecar with a populated tags: list, which write() short-circuits as unchanged and therefore never touches. It could not have failed. Meanwhile tags: and provenance: written with nothing under them were being rewritten to tags: [] and provenance: {} on every pnk link — two lines changed outside the block, in the increment whose test says none are, against a promise stated as byte-identity. Reachable before L6 only from a paid PDF extraction, which is why L5b's sweep missed it; pnk link reaches it on a first link, the common case. The sibling test_a_known_key_with_a_null_value_does_not_crash_the_writer parametrises exactly these three keys and asserts only "id:" in text: it pins the absence of a crash and nothing about the value.
• The embedded-NUL test put its NUL in the filename — docs/a\x00b.md — where only the parent is resolved, so it never reached the guard it was written for and passed against the ordinary "not a document" refusal. Moved into a directory component. Caught by mutation, not by review.
• assert "outside" in message against a fixture named outside.md, and assert "partner" in message against a tmp_path ending in /partner. Both were satisfied by the interpolated path, so the reason could have vanished from the wording with the test still green — proven by rewording the error and watching all 29 pass. Fixtures renamed, phrases asserted.
• A fixture stops reaching its guard when a later fix gets there first, and nothing says so. The ordering test for the containment check was retargeted twice — once when the static refusal was added, once when that learned to resolve the prefix — because each fix caught its input earlier, leaving the test green and its guard unexercised. Both times the mutation found it and the reading did not. Re-run the whole mutation battery after every fix, not only a mutant for the fix itself: a fix can silently disarm a test written for something else.
• The test written for the freshness branch — the branch a finding had just called untested — asserted only report.ok, which holds whether that branch runs or not. Proven by forcing is_stale to return True: the branch never ran and the test still passed. A skipped-fresh row carries no issue, so link_scan is the assertion that discriminates. Found by a reviewer, not by the round that wrote it, in the commit whose message called its other two fixes mutation-verified — "mutation-verified" is a per-assertion claim, not a per-commit one.

A docstring claiming a safety property its function cannot have

MEDIUM. _doc_id_of's owner argument was documented as preventing the pnk://self/… retargeting defect. It cannot: only .id is returned, so owner never reaches an observable — measured both ways, the mutation is caught by no test and the output against a partner sidecar carrying the exact retargeting shape is byte-identical. The protection is real but lives in linkscan.scan_one, which keeps the links it reads. A plausible rationale attached to the correct line is harder to catch than a wrong line, because reviewing it means re-deriving the claim rather than reading the code.

Mutation testing: a killed run poisons everything after it

HIGH, methodological. The first mutation run blew a two-minute timeout and was killed mid-mutation, so its finally never restored the source. The next run's pattern then failed to match the already-mutated file, reported "pattern not found", skipped — and that guard stayed disabled for all ten mutants that followed. The signature is unmistakable once known: one unrelated test failing on every mutant, including mutations that cannot reach it.

Two things made it recoverable: the disabled guard had its own test, so the failure was loud, and ./check.sh had been green minutes earlier, which dated the contamination. The fix is a baseline snapshot taken before the first mutation and asserted after every restore — not git diff --quiet, useless in the increment's own worktree where the source is legitimately dirty. Scope the run to the modules under test, too: the full-suite run is what blew the timeout that caused this.

Every fix was mutation-tested against the test written for it, and three escaped, each in a different way that "green" could not distinguish. The NUL guard had a test whose input never reached the line. The containment-ordering test stopped reaching its branch twice, when a later fix caught its fixture earlier. And review 14's next() guard had no test at all: the "" and "." written for it raise at the glob() call, not at the step, so the guard one line down was never executed — which review 14's own commit message called "eleven mutants, each killed by the right test", and this paragraph called "all but one".

The method is now: mutate every behaviour in the function, not the ones the diff touched. That standard was stated before it was met. The sweep that first claimed it covered three behaviours; an independent 47-mutant pass over the whole of sidecars_under and scan_one killed 33 and left 14 alive — 2 provably equivalent, 12 unpinned. Every one of the 12 was checked and the code is right in each, so they are coverage rather than defects, and they are listed rather than closed: the two halves of the exclude disjunction (a deliberate mirror of sync._excluded), the continue that bounds a pre-walk escape, .resolve() on anchor and on base, the is_file() and sidecar-suffix skips, the two sorted() calls, partner_sources raising, and LinkTargetMissingError's count.

Naming them is the point. A number for the battery is unverifiable afterwards — the runs leave no artefact — but which behaviours are unpinned is checkable by anyone who repeats the sweep, and that is what a later reader needs.

One mutant is genuinely equivalent: substituting the locally declared [[links.kb]] id for the partner's own when writing an alias target changes nothing, because the refusal above has already established the two are equal. Saying so is part of the result — the rule is enforced by that refusal, which is caught, and the docstring records it so nobody simplifies the variable away on the grounds that they are the same.

Green expires at the next keystroke

HIGH. ./check.sh ran green, then a docstring was reworded to 101 characters, then the increment was committed. Under set -e a failing ruff check means the eleven gates after it never ran either: the increment's own verification stopped at gate two, unnoticed, because the earlier green run was still in mind. The rule already says green-before-review; what this adds is that the run has to be the last thing before the commit, including after an edit to a comment.

A containment rule argued in prose and implemented for half its inputs

HIGH. sidecars_under reads a partner's [sources], and its docstring says why that input is untrusted: "without the same check here, a partner manifest could point the walk at any directory on this machine, and roots = ["/"] would be an unbounded walk on a post-commit hook." The check existed for roots. include is exactly as partner-controlled and had none, so include = ["../../outside/*.md"] walked out of the partner KB and this one recorded inbound links from files the partner does not own — complete true, so sync persisted them.

The line that looks like the guard is not one. candidate.relative_to(root) ran on every match, and relative_to is purely lexical: docs/../../outside/planted.md is relative to the root as a string, returning docs/../../outside/planted.md rather than raising. A .. is only collapsed by resolving, which is what the roots branch does one block above and this one did not. Two spellings of the same rule, ten lines apart, one of them not implementing it.

The fix then took four goes, and every wrong one came from spelling the rule differently from the place that already had it right. link._document_in resolves the parent and leaves the final component: the directory chain is followed, so .. collapses and an escape through a symlinked ancestor is caught, while the document's own symlink is irrelevant. That is the rule. Each attempt reinvented it:

1. Per candidate, after globbing. Correct about what to refuse, but it refuses the results: glob has already enumerated and stat'd the whole tree by the time the first match is inspected, so include = ["../../../../**/*.md"] still walked the machine on every post-commit. And an escape sets complete false, so no last_scan is written and the TTL cannot suppress the retry either — unbounded, forever.
2. Refuse any pattern containing .., before globbing. Bounded, and wrong in the other direction: ../notes/*.md stays inside the KB and the partner's own walk_sources ingests it. This KB called a legitimate manifest an escape and then never refreshed that partner again. Refusing a partner's valid configuration is the same defect as accepting an invalid one.
3. Resolve the prefix before the first glob component. Defeated by a pattern that starts with one: */../../../outside/**/*.md has an empty prefix, so the check passed unconditionally and the .. ran inside glob — attempt 1's defect, reachable again. It also refused a fixed pattern naming a symlinked document, because with no glob component the "prefix" is the whole path and resolving it whole follows the final symlink: include = ["alpha.md"] refused while include = ["*.md"], reaching the same file, was accepted.
4. Join the whole pattern and apply _document_in's spelling to it. A glob component is just a name that does not exist, which resolve() collapses lexically, so one resolve() answers it with no enumeration. Ten patterns measured — a .. staying inside, a directory genuinely named a..b, a literal bracket, two escapes, one behind a leading glob, a symlinked directory under a glob, a symlinked document by both spellings, and an absolute — all correct, escapes refused in 0.12ms without touching a 3000-file tree. None of the ten contained ** followed by ...
5. Drop ** from the probe. ** matches zero or more components while Path.parts counts it as one, so keeping it let a following .. cancel it and the probe landed one level below where the walk goes. **/../../**/*.md probed inside the KB and walked the directory containing it, recursively — linear in the outside tree, and silent, because an escape is only noticed once a candidate is yielded and that pattern matched none. Dropping it is exact rather than merely conservative: each component ** expands to is one a following .. then pops, so the zero-expansion is the highest the walk can reach.

Attempt 4's measurement was real and its ten patterns were all correct. It was the sampling that was wrong — ten hand-chosen inputs, none of which combined the two tokens whose interaction is the whole difficulty. A table of cases proves the cases in it, and reads like proof of the rule.

An absolute pattern is refused separately, because glob cannot walk one wherever it points — including at this KB's own docs/. It had been folded into the escape message, which was simply false for that case.

The lesson is not about paths. Three of the four attempts were written by reasoning about the problem afresh instead of copying the spelling from the function twenty lines away that had solved it. A rule implemented twice is a rule with two behaviours; the fix was to make the third implementation textually identical to the first two and say so in all three.

Each attempt was found by mutating its predecessor, and three of them disarmed an existing test: a fix that catches its input earlier leaves the older test green with its guard unexercised. By the last round two guards written in earlier increments had gone dead this way — L2's roots containment check, whose test was satisfied by the substring "outside the KB" that the new per-candidate check also emits, and scan_one's own except around the walk, whose only input (include = ["/etc/**/*.md"]) the absolute branch now answers first. Both were found by mutating behaviours this increment never touched.

So the rule is stronger than "re-run the battery after every fix": the battery is over the whole function, not the diff — and a promise worth a guard is pinned directly (here, by making the walk raise) rather than through an input that some later fix can intercept. Running it that way in the next round found three more unpinned behaviours: the sidecar existence check, which predates L6 (7570a69), and the */** boundary and the next() guard, both written two rounds earlier by this increment (425d106). The sharper reading is the second one — the code least likely to be pinned is not the oldest, it is what a recent fix added while attention was on the defect it closed.

The fix for "one bad entry is not the end of the partner" was itself incomplete, one line either side of where it landed. probe.parent.resolve() sat above it unguarded, so an embedded NUL in any but a pattern's final component still raised out of the function; and Path.match("") sat below it, so an empty exclude entry did the same — a case the new guard's own comment cited by name as its reason for being scoped tightly, and then did not handle. Both produced the outcome the commit had just declared impossible. A guard placed by reasoning about one call is a guard for one call; the same mechanical sweep that closes an escaping-error class — list every call that can raise, ask what each does with bad input — is what this needed, and it is the third time in this increment that lesson has been relearned rather than applied.

One more from the same sweep: the containment predicate was fooled by a trailing .., because Path("/kb/..").is_relative_to("/kb") is lexically true. The final component is left unresolved so a symlinked document stays readable, and .. is never a document.

sync.walk_sources has the identical shape for the local manifest and is not fixed here: it is the user's own configuration rather than a partner's, and changing the engine's document walk is not this increment's to do. Reported instead, and now scoped as its own increment and PATCH release in plans/20260731_2128-source-walk-containment.md — which measured a third defect this pass had not: an absolute local include is a raw NotImplementedError out of cli.main, the same escaping- error class L6 spent four passes closing on the partner side.

Smaller things

• pnk link A A wrote a self-loop, which says nothing and would return the document as its own neighbour. Refused now — and worded for both ways of arriving, because only the ULID is known here: the target really is this document, or it is a different file carrying the same id, which is a KB fault in its own right. "would link to itself" told someone who had named two different documents that one of them was itself, and pointed at neither the duplicate nor pnk doctor, which finds it.
• os.replace onto a symlinked sidecar destroyed the link and left a regular file, with the real file elsewhere still holding the old text. create() guards this explicitly; write() did not, and pnk link is the first command a person points at a file of their own choosing. It now writes through the link.
• The error fallback named the local KB root while the comment beside it claimed it named the declared path — reporting an unrelated readable directory for a failure that had nothing to do with it. Neither of the two tests written for that fix caught it: both asserted only the message prefix.
• why_not_a_kb reproduced, one level down, the defect its own docstring exists to record. The three-way split was added because an is_dir() split called an existing regular file "no such directory" — "the one answer a person would check and find false". But the caller's probe is is_file(), so a pinakes.toml that exists and is a directory, or a symlink to nothing, fell through to "no pinakes.toml there" with the file plainly visible in ls. Found in review 9 by reading the docstring's justification against the code beneath it, which is a cheap check worth running on any function whose comment argues for its shape.
• pnk link takes no lock, so a concurrent write to the same sidecar can lose one side's change. Rename-atomicity prevents a torn file, not a lost update, and DESIGN §2.2 now says which.
• STATUS's surface you can use today table had no pnk links row at all, an hour and a quarter after it shipped in 0.5.0 (20260731 11:27; the row landed in b96d247, 12:44) — found while writing the increment by reading the neighbourhood rather than the diff.

L7 — pnk doctor's link checks (20260801 05:40)

The check read a partner's index, and DESIGN §6.2 forbids exactly that

HIGH. The cross-KB check opened <partner>/.pinakes/index.db read-only to ask whether the target document exists. §6.2 rules that out in the sentence that defines reverse links: they come from the other KB's committed sidecars, "not its index, which is gitignored and simply absent in a fresh clone, and which could not be read without holding a second KB's lock" — repeated verbatim in linkscan's module docstring, which is the module the check imports from.

mode=ro is not enough, and this is the part worth keeping. Measured: a read-only connection still materialises index.db-shm and index.db-wal inside the partner's .pinakes/, and cannot checkpoint them away on close. A diagnostic command wrote into a KB it was only asked to look at. Two more consequences fell out of the same choice: a partner cloned but never synced answered "missing" for every target, and a partner whose .pinakes/ is mode 0500 degraded silently with an internal StoreError message that misdiagnosed the cause.

The fix is the machinery L2 already had — partner_sources + sidecars_under + read_sidecar — which is design-conformant, works on a fresh clone with no index at all, and is now tested that way.

The rule was in the imported module's own docstring. Not a subtle design point: a paragraph in the file the new code imports three names from.

The metric's numerator and denominator came from different populations

HIGH. Coverage is COUNT(DISTINCT src_doc_id) / active. sync's SoftDelete sets state = 'deleted' and drops the chunks — it never deletes that document's origin = 'sidecar' rows. So a deleted document still counted toward the numerator while leaving the denominator, and the headline number of this increment reported 2 of 1 documents linked (200%).

A ratio built from two queries is two populations until something makes them one. The join is one line; noticing it needed one was the work.

The declared [[links.kb]] id is not evidence of which KB is at that path

MEDIUM. The check keyed partner document sets on linked.id — the local declaration. linkscan.scan_one refuses that substitution with LinkedKbIdMismatchError, and DESIGN §6.2 rule 1 states it as a rule, because trusting the manifest files another KB's links under this alias. Measured both ways with a manifest declaring X over a partner whose real id is Y: a target that existed in Y was reported unresolved, and one that did not was silently resolved.

Two directions need two tests, and only one of them is obvious. Filtering on the declared id also skips a partner whose real id is the one wanted — a dangling target that goes unreported rather than misreported — and that mutant survived until a test was written for it specifically.

Four remedies could be blanked with the suite green

MEDIUM. The plan required "every new WARN carries a remedy" precisely because the meta-guard (test_every_problem_carries_a_remedy) runs on a fixture where these checks are OK and carry no problem. The helper written to stand in for it asserted is not None — which "" satisfies, while the guard it substitutes for asserts truthiness. Four of five remedies were emptiable.

A stand-in for a guard has to assert what the guard asserts. It now returns the string and each caller asserts a phrase from it.

A test named for a guard, authoring nothing that reaches it

MEDIUM. test_an_unreadable_linked_kb_path_is_a_warning_not_a_traceback was written for the sentence "a diagnostic command reporting a traceback is the one outcome pnk doctor may not have", and named why_not_a_kb's "third caller needing the same try". It authored no cross-KB link -- so wanted was empty, _unresolved_cross_kb returned before touching the partner, and the test pinned the guard in _linked_kbs and neither of the two in the function the review had just added. Both are load-bearing: a partner directory behind a mode-0000 parent raises PermissionError out of partner_sources, and a roots entry carrying an escaped NUL -- which tomllib accepts and Path.resolve does not -- raises ValueError out of sidecars_under.

Third time in two increments that a fixture stopped one step short of its guard, and the shape is always the same: the test sets up the failure but not the demand for it. An unreadable partner is only reached by code that has a reason to read it.

The dangling-link side of the soft-delete interaction had the same gap in miniature -- the fixture that proves the numerator excludes a deleted document already produced 1 dangling inside this KB in the detail it held, and asserted nothing about it. The fix was one line in a test that already existed.

Mutants that were not the logic they claimed

Methodological. Four "blank the remedy" mutants replaced "A cross-KB target…" with "" or "A cross-KB target…", which evaluates to the original string. All four reported SURVIVED, which read as four coverage gaps and was really one broken harness. Rebuilt to replace the whole remedies.append(...) call, all four die.

This is the second increment where a mutant that did not reproduce the real prior logic was briefly taken for a result. The check is cheap: a mutant that survives should be run against the case it claims to break before it is believed.

G2 — The headroom measurement, and what it found (20260801 12:14)

G2 was built to answer one question: is the graph release's gate reachable at all? It is not, on this corpus. The precondition needed at least 7 of the ~18 single-KB multi-hop questions to fail today. One fails. G3 does not start, and the answer arrived before anything bumped schema_version and forced every KB in existence to rebuild — which is the whole reason the measurement was sequenced first.

HIGH — the demo corpus has no tags, and one directory. The plan assumed otherwise. tests/demo-kb is thirty documents in a flat docs/, and not one sidecar carries a tags key. With mentions cut (decision 6), that leaves exactly one structural edge kind that crosses a document boundary: co-located, through a single thirty-way directory hub. shared-tag derives zero edges. sibling, parent/child and in-section are all intra-document and cannot bridge two evidence documents by construction. So the "derived structure" the graph release exists to evaluate is, on the committed corpus, one hub — and G5's own text reasons about "the directory layout and tag vocabulary of tests/demo-kb" as though a vocabulary existed. It does not. Whatever G3 would derive here, a result carried by it is a claim about one directory.

HIGH — a reachability probe on a thirty-document corpus is close to vacuous, and the reason is not the probe. candidates_per_source is 30 and the corpus has ~30 chunks, so the vector channel already returns essentially every document with a positive cosine: the funnel sees the whole corpus on every query and then cuts to final_k = 5. A failing question is therefore almost never a recall failure the channel could fix by reaching further — it is a ranking failure. That is why the probe reports at-seed separately from liftable: two of the three questions the fake backend called liftable were already among the fused candidates and merely ranked below the cut, having traversed no edge at all. A ceiling built from those would have read as headroom and been none.

The numbers, real [light] models, tests/demo-kb at 20260801 12:14. 18 multi-hop questions, 1 failing (mh-withdrawn-collection-register), liftable 1 without authored edges and 1 with, beyond-2-hops 0, membership-only 0. Required: 7 failing and 7 liftable without authored edges. It fails on the first clause by six.

The questions were frozen before the probe ran, and were not re-authored afterwards. Thirteen new multi-hop chains, authored from pairs of documents that between them answer one question and share no vocabulary on the thing that joins them, with the second hop phrased in the first document's words. Seventeen of eighteen are answered correctly. Re-authoring them until seven fail is fitting the question set to the gate — the circularity decision 14 removed by cutting cross-KB questions, and undetectable once done. The honest reading is that a corpus of thirty short, topically disjoint documents cannot produce a hard multi-hop set: picking 5 of 30 is not a discriminating retrieval task, and the pipeline scores 0.94 on it.

MEDIUM — the fake backend and the real models disagree about the shape of the answer, and only one of them is the measurement. Under the deliberately tie-heavy hashing fake the same set shows 9 failing and 3 liftable without authored edges (6 with) — the exact with/without gap the plan predicted L1's hand-authored links would produce. Under the real models both collapse to 1. A measurement taken on the fake would have reported a different failure of the precondition and invited the wrong remedy.

MEDIUM — _score read Outcome objects, so the artifact could not have been re-scored. Every metric is a function of five fields per question, but the scorer was written against the in-memory type. Splitting score_rows(rows) out is what makes the committed artifact checkable offline, and it is what test_the_committed_41_score_exactly_their_pre_growth_values runs on: no weights, no network, and the 41 pre-growth questions reproduce their baseline byte-identically — measured on macOS against a baseline written by CI's ubuntu runner, which is the same cross-machine agreement G1's new CI job independently confirmed the same morning.

LOW — the first --fake run silently asked for real weights. _fake_kb asserted each manifest substitution appeared exactly once; provider = "fastembed" appears twice (embedding and rerank), so the assertion fired and the run aborted — correctly. Loosening it to "replace whatever is there" would have left the rerank provider real and made an "offline" gate download a model. The expected occurrence count is asserted per line, not assumed.

Review pass — MEDIUM, the empty-set skip could swallow a typo. load_questions read raw.get("questions") or [], so a file whose key was misspelled produced an empty list — which the new skip then reported as "a template deliberately ships none" and exited 0. Under the old behaviour that file failed. An absent key is now an error and only an explicit questions: [] skips; the two cases are genuinely different and the skip is only safe because it can tell them apart.

Review pass — MEDIUM, read_outcomes promised more than it delivered. Its docstring said it "refuses a file whose rows are not rows — never a partial read", and a row missing confidence raised a bare KeyError from the middle of the loop. Every one of the five fields reaches a metric in score_rows, so a row missing one cannot be scored; they are now checked by name.

Review pass — LOW, fused_candidates is a stage, not an entry point. It does not run check_coherence, because search does that before calling it. Anything reaching for the new public function directly is querying an index that may have been built by a different embedding model, which returns confident nonsense rather than an error (§4.4). Stated in the docstring rather than duplicated in the function, since search would then run it twice on every query.

Source-walk containment — one rule, three sites, enforced at one (20260801 13:28)

The durable lesson: a containment rule argued in prose beside one of its two inputs is a rule for one input. manifest._sources states that a source root must stay inside the KB and enforces it for roots. include sat two lines away, validated nowhere. The same lexical candidate.relative_to(root) non-guard then appeared at three sites — linkscan.sidecars_under (fixed in L6 review 10), sync.walk_sources, and the sidecar sweep beside it — and the one whose docstring carried the argument was the one that did not implement it.

All three defects were re-measured on 0.7.0 before anything was changed, against a plan that had measured them on 0.5.0 at 900aae7. All three still reproduced, unchanged:

	Before	After
include = ["../../outside/*.md"]	2 indexed, a sidecar minted outside the KB, document keyed docs/../../outside/secret.md	ManifestError at load, naming the pattern and the root
include = ["/abs/path/*.md"]	bare NotImplementedError traceback out of cli.main	ManifestError: "is an absolute path", with its own remedy
docs/escape -> /outside, include = ["*/*.md"]	1 indexed, a sidecar minted outside the KB	0 indexed, the pattern reported, nothing written outside

HIGH — a fourth defect, found by a test that was meant to pin correct behaviour. Layer 1 deliberately accepts a .. pattern that lands inside the KB (include = ["../notes/*.md"] from docs/), because what matters is where a path lands rather than whether .. occurs in it. The test asserting that then failed on the document's key: relative_to is lexical, so it returned docs/../notes/n.md. Measured with roots = ["docs/", "notes/"] and include = ["../notes/*.md", "*.md"], one file on disk produced one indexed document and two failures — "appeared after the walk had already read this directory" — because the sidecar found under one key was invisible under the other, and the unmatched sweep reported an indexed document as unmatched. Nothing in the plan predicted this; it exists only because the legal .. case had never been exercised.

The fix is lexical collapse (posixpath.normpath), not resolve(). Resolving would follow a symlinked directory and re-key every document under it — docs/alias/x.md becoming docs/real/x.md — which on an existing KB is a path change against a permanent identity. Lexical collapse touches only paths containing .., and every one of those is already broken today. Containment does not rely on it: the per-candidate check resolves.

The predicate was copied from linkscan.sidecars_under, not re-derived, and that was the whole point. Reviews 11, 12, 13 and 14 each found a different defect in a different spelling of this one rule: refusing any .. (rejects a valid manifest); resolving only the prefix before the first glob component (defeated by a leading *); resolving the whole path (refuses a symlinked document while accepting the same file via *.md); and keeping ** in the probe (it matches zero components while Path.parts counts it as one, so a following .. cancels it). Re-deriving would have cost that sequence again for nothing.

MEDIUM — the static layer is the bound, and the dynamic layer is the guard; neither covers the other. Checking candidates after globbing refuses the results while still paying for the enumeration, which is what the roots rule exists to prevent — test_an_escaping_pattern_is_refused_without_enumerating_the_tree counts entries pulled from the generator, not resolve() calls, because the cost being avoided is the walk itself. And a symlinked directory has no .. and no absolute path, so it is invisible to any load-time check. The per-candidate test breaks rather than continues, and runs before the is_file()/sidecar skip: a pattern reaching outside that matched only directories or only sidecars hit one of those continues first, so the walk left the KB and reported nothing.

LOW — the default include is safe by luck, not by design. ["**/*.md", "**/*.txt"] does not escape through a symlinked directory, because pathlib's recursive ** skips them. Any user who writes a non-recursive pattern loses that, which is exactly the shape of a guarantee nobody knows they are relying on. Stated in walk_sources' docstring rather than left as folklore.

Mutation round — three survivors, two of them defects (20260801 13:38)

Eleven guards broken on purpose. Eight were caught immediately. The three that survived were worth more than the eight:

HIGH — the per-root skip copied from linkscan was data loss here. sidecars_under does if pattern in escaping: continue, so a pattern known to escape contributes nothing under any later root — correct there, where a dropped candidate costs one inbound link and a partner's [sources] is one statement about one KB. Copied into walk_sources it means something else entirely: the escapes this loop can see are symlinks, which are a property of one directory rather than of the pattern, and a dropped candidate here is a deleted index row and an orphaned sidecar. So docs/escape -> /outside silently stopped */*.md collecting anything under an unrelated second root. Removed, with a test. "Copy the predicate, do not re-derive it" was the right instruction and this was still the wrong thing to copy — the predicate and the policy around it are different decisions.

MEDIUM — the containment check ran before is_file(), and no test could tell. Every symlink test matched a file, so moving the check after the skip changed nothing observable. The case the ordering exists for is a pattern that matches only a directory (or only sidecars): it hits that continue first, and the walk leaves the KB reporting nothing. */* against a symlinked directory containing a subdirectory is that case, and it now has a test.

MEDIUM — the break bounded nothing, because sorted() had already drained the generator. The plan carried break, not continue, on a 360× measurement from linkscan review 12 — where the loop is lazy. Written here as for candidate in sorted(root.glob(pattern)) the enumeration a symlinked escape triggers has already happened by the time the first candidate is inspected, so the break saved only the loop body, and the resolved cache made even that one dict lookup. This is the shape of a guard inherited with its justification and without the property the justification rested on. The loop is now lazy; output order does not depend on it, because walk_sources sorts what it returns and the per-root sort only decided which of two candidates sharing one key won — and they describe the same file with the same hash. Measured: 301 entries enumerated before, 1 after, and both the break and a reversion to sorted() are caught by that number.

Two tooling corrections swept in the same PATCH (20260801 13:52)

tools/link_density_gate.py resolved one of its two bases and not the other, so any non-canonical root — every /tmp path on macOS — exited with a traceback. One root.resolve() at the top of census, and a test driving the tool through a symlinked parent.

tools/fragments.py's duplicate-heading defect was already closed, and the open-corrections entry saying "the tool is unchanged" is stale. Measured rather than assumed: three fragments (two fixed-*, one added-* whose body begins - **Fixed: …**) spliced into a section that already had a ### Fixed produce exactly one ### Fixed, one ### Added, and the category-prefixed entry filed by its filename, which is where the category belongs. Both halves have regression tests already — test_fragments_merge_into_a_category_heading_that_already_exists asserts count("### Added") == 1. Closed by the 0.6.0 release-prep commit, not by this one.

LOW, and the reason to write this down: fragments.py --apply is anchored to the repo it lives in, not the working directory. Testing it by cd-ing to a temp tree spliced this worktree's CHANGELOG.md and deleted its changelog.d/ fragment, reporting success. --repo exists exactly for that and the tool's own test suite uses it. The damage was recoverable only because the fragment had already been committed — which is the same rule the G1 mutation harness earned: commit before running anything that rewrites the tree. A git checkout -- to undo a mutation in the same session then reverted an uncommitted fix in tools/, for the second time in this project, and was caught only by re-reading git diff --stat afterwards.

The reachable-ceiling probe, against a corpus it did not ship with (20260804 04:21)

HIGH — a measurement tool that absorbs malformed input reports a number that looks valid. The finding class, stated once because it generalises past this tool: every defect below is an input the probe accepted, turned into a plausible verdict, and reported with no mark on the output. That is strictly worse than a crash. A crash costs an hour; a number that is quietly wrong is read into docs/STATUS.md, decides whether a schema_version bump is licensed, and is not falsifiable after the fact — nobody re-derives a measurement that already looks fine. Anything that converts input into a number owes its caller a refusal for input it cannot measure, and the refusal must be a named failure, never a diagnostic line a reader has to notice.

The two found by the rehearsal that ran the probe against an external KB — both measured on demo-kb under the offline fake backend, where the corpus reads 18 multi-hop / 9 failing / 3 liftable (the real [light] reading of the same corpus is 18 / 1 / 1, so the real-model impact of each is several times larger):

• A hop expect naming a path not in the index resolved through a lookup that answered "" for an unknown path, so the hop was recorded lands=False, reachable=False — failing and unreachable, identical in the output to a genuine one. One typo took failing from 9 to 10 while liftable stayed 3. On a 200-document corpus converted by hand from a frozen question set, this is not hypothetical.
• A multi-hop question with no hops incremented the denominator and produced no verdict, so it could never be counted failing and appeared in no other figure: 18 became 19, invisibly. The scaffolded template documented id, question, expect and kind and never mentioned hops — the trap was armed by our own template, which is why the fix edits both the tool and src/pinakes/templates/notes/eval/questions.yaml.

MEDIUM — the first fix was narrower than its own commit message claimed, and an adversarial pass found three more of the same class. Worth recording because the pattern is the lesson, not the individual bugs: a guard written against the two known instances validated the thing the bug report named rather than the property the measurement needs.

• A document with no chunks. The guard asked whether the path was in documents. Every node the channel walks is built from the chunks table, so a document with zero chunks — a blank file, a note that is only front matter, a PDF whose free extraction yielded nothing — passes a path check and is still incapable of landing or being reached. It reproduced defect 1 digit for digit (failing 9 → 10, liftable 3), from a path spelled correctly. "The name resolves" is not "the measurement can use it".
• A multi-hop question with exactly one hop, which the guard's own wording called "multi-hop in name only" and let through. This one is worse than the defect it was written for, because it moves liftable upward (3 → 4). Under-counting fails safe against a floor; over-counting licenses the schema bump. A guard on a threshold must be written in the direction that can do harm, and the harmful direction here was the one nobody had an example of.
• An empty hop query, absorbed the same way, and a golden set with no multi-hop question at all, whose entirely-zero report is indistinguishable from a measured one.

MEDIUM — the second review pass found the same hole again, in the key nobody had looked at. question.filters is applied to the last hop and was never validated. A tags, path_prefix or source_type the index does not hold makes that hop unable to land whatever the corpus contains. Measured on demo-kb under the fake backend, against its 9 failing / 3 liftable: one such filter took failing to 10; the same filter across every multi-hop question took the run to 18 failing / 0 liftable, exit 0, unremarked. (The review pass that found it quoted only the second figure for a single question — checking it is what caught the difference, which is the M5 lesson applied to the fix's own write-up.) It is the empty-query defect wearing a different key, it moves both binding clauses, and failing moves upward. Two review passes each found one more instance of a class the first fix was supposed to have closed, which is the actual lesson: the guard has to be written from the list of everything the measurement consumes, not from the list of bugs already reported. What probe() consumes is now the checklist — hops, each hop's query and expect, the document behind that path, and filters. It is validated through search's own _filter_sql rather than a hand-written copy, so the check cannot drift from the semantics it is checking.

MEDIUM — a third pass, and the same lesson a third time: the artifact recorded every setting except the one that moves the number most. retrieval.rerank records the mode (local), never the reranker's provider and model — and lands is expect in the top final_k after reranking. Demonstrated by the reviewer on one corpus, one path, one manifest, with only [rerank] model changed: 9 failing / 3 liftable became 18 / 12, and every identifying field in the two artifacts compared equal. Worse, the commit that added the block claimed it mirrored eval.py::_header — which carries three blocks, embedding, rerank and retrieval, its docstring saying it holds "every setting that can move a row". The copy took two of the three and dropped precisely the one not derivable from the others. When you cite a prior art as the standard you met, diff against it. index_built_at joined the payload at the same time: a corpus edited since its last pnk sync is measured as it stood then, and nothing else would say so.

LOW, and the most human of the findings: one defect, two accusations. The filters check ran before the hop-path check, and filters cannot admit a path the index does not hold — so a mistyped expect under a healthy filters: block produced two problems, the first of them pointing at the wrong line, and a {len(problems)} count that overstated. Ordering between checks is part of a refusal's correctness, not a detail: the message that names the wrong cause costs the same debugging hour the guard was written to save.

LOW — a sentence assembled from parts is a sentence nobody read. The per-kind wording was spliced mid-clause into three messages, and on the branch no test covered — a non-multi-hop question carrying hops, which load_questions allows — it rendered "so this probe never measures this question — only multi-hop — so no figure moves for the query rather than for the corpus — the same silent deflation as a mistyped path": two sos, and a closing clause asserting the deflation the same sentence had just denied. The commit message claimed that wording was fixed; no test exercised it. Each message now ends with a whole sentence, and a test covers the branch.

MEDIUM — a test can pin a claim it cannot falsify. test_the_probe_names_the_kb_it_measured ran only --fake, and --fake measures a copy of the demo KB: every assertion in it was satisfied by a probe that ignored --kb and hardcoded the demo path, which is the very defect the test names. It now runs a real --kb against a KB deliberately not called demo-kb (a small runner script registers the fake backend in the subprocess). A test whose fixture is the default cannot detect "always reports the default".

The same mistake then repeated one layer down, and is worth recording because it is so easy to miss twice: the replacement asserted that kb_root is resolved, using tmp_path — which is already absolute and already resolved, so dropping the .resolve() left the entire suite green. An assertion whose fixture already satisfies the property tests nothing. It now also runs with a relative --kb from the corpus's own parent directory, which is the only shape that can fail.

MEDIUM — naming the corpus is not naming the measurement. The artifact recorded which KB was measured and not what produced the numbers. failing is hop.expect in the top final_k passages, downstream of candidates_per_source, fusion, fusion_top_k, rerank and vector_tier — all per-KB manifest keys. Two artifacts from two configurations of the same corpus were indistinguishable, the exact defect the KB-naming fix was written to close, one level in. eval.py's artifact header already recorded the same set for the same reason; this one now does too.

MEDIUM — quoting a number without its backend. The first commit message and changelog fragment said "measured on demo-kb: failing 9 → 10" without saying the numbers were the hashing fake's; this repository's own retrospective already records that the fake and the real models disagree about the shape of that answer, and the real reading is 18 / 1 / 1. A user-facing fragment reads as the real measurement. Every measured number carries the configuration that produced it, or it is a different claim than the one intended.

MEDIUM — four passes, and the identity question kept moving outward one input at a time. Pass one: the artifact did not name the corpus. Pass three: it named the corpus but not the pipeline. Pass four: it named corpus and pipeline but not the golden set — the input every printed figure is computed from, and the one this branch's own refuse-edit-re-run loop changes most often. Demonstrated on one corpus, one index, one manifest, rewriting only the hop queries into a generic word: 9 failing / 3 liftable became 18 / 9, and every recorded field except reports compared equal. The payload now carries the golden set's resolved path, a sha256 of its bytes and its counts, plus revision on both model blocks — a revision selects weights as surely as a model name does. One correction the sixth pass earned: that "needs no re-sync, so nothing else would move with it" is true of [rerank] revision, which nothing compares against the index, and false of [embedding] revision, which search.check_coherence guards — change it without a re-sync and the run stops rather than drifting. Both are recorded; only one of them could ever have moved a figure in silence. The general form: an artifact must identify every input its numbers are a function of, and the way to find them is to enumerate the function's arguments, not to wait for a reviewer to name one.

LOW — the contradiction moved instead of leaving. The per-kind wording was fixed once by appending the conditional sentence to the end, which left "the hop is recorded failing-and-unreachable. No figure this probe prints moves" — an assertion and its denial, one sentence apart, in a message the previous commit claimed to have fixed. The consequence is now entirely inside the conditional (_consequence), so a non-multi-hop question is told nothing was recorded at all, and the test asserts the class ("no line may claim a hop was recorded") instead of one superseded string.

LOW — one more absorption, found by asking what check_measurable does not compare. Every hop was validated on its own and never against its siblings, so two byte-identical hops passed: MIN_HOPS satisfied, one retrieval written twice, and liftable moved from 3 to 4 on demo-kb — upward again. A YAML copy-paste is the realistic route.

MEDIUM — five passes, and the fixture-satisfied assertion came back twice in one commit. The pass-four commit added the golden set to the artifact and asserted it only under --fake — where the measured golden set is demo-kb's, so hardcoding the demo path and digest passed every assertion and the full suite. That is the identical defect pass two found for kb_root, recorded in this very fragment as "a test whose fixture is the default cannot detect 'always reports the default'", reintroduced by the same author two commits later for the input he had just added. The same commit pinned revision on both model blocks against manifest.<section>.revision — and demo-kb declares neither, so both assertions were None == None. Writing the lesson down does not apply it: the check is mechanical — for every assertion, ask what value the fixture already has, and whether a hardcoded constant would pass. _fake_kb now writes distinctive revisions into its copy, and the golden-set identity is pinned on the real---kb run.

LOW — one more absorption, one normalisation short. The identical-hops check compared (query, expect) byte-exactly, so upper-casing or padding the duplicated query defeated it while the retrieval stayed identical — FTS5 folds case, every backend here splits on whitespace. Measured on demo-kb: liftable 3 → 4, exit 0, the same upward move the check was written to stop. The fingerprint is now case-folded and whitespace-collapsed. A guard on "the same input" must normalise the way the consumer normalises, which is the _filter_sql lesson again in a smaller key.

The fix removes the place the defect could live, not just the symptom. _doc_id is gone; check_measurable validates the golden set against the active documents rows and the chunked subset up front, and probe is handed the resulting map, so an unknown path has exactly one place it can be handled and that place refuses. Validation runs before the backend loads — on a real run that is a model download, and a run that is going to refuse should refuse in a second.

Two smaller defects of the same family. --fake silently discarded --kb, so --kb <corpus> --fake measured demo-kb and reported its numbers as the corpus's; and neither output format named the KB, so two runs against two corpora produced artifacts indistinguishable on inspection — which is exactly what made the discarded --kb survivable. The pair belongs together: a silent substitution is only dangerous because the output is anonymous, and naming the input in the artifact is the cheapest defence a measurement tool has. The closing prose's hardcoded >= 7 was the same error in prose form, a claim about one corpus printed under the numbers of another; the threshold now stays with the corpus's own measurement plan.

On testing a refusal in a subprocess. These tests run the probe against a KB whose manifest names a backend the test subprocess never registered, so a run that got past the refusal fails too — a bare non-zero exit proves nothing. Every refusal test asserts the named message and the offending id/path, and test_a_well_formed_golden_set_is_not_refused is the control that keeps the message attributable to the question rather than the environment. Two assertions were weak for the same reason and were tightened: "hops" in stderr is contained in every refusal's closing remedy, and "--fake" in stderr is satisfied by any argparse error, since the usage line names both flags.

One deliberate over-reach, recorded so it can be overruled. A question-level expect naming a missing document refuses the run although probe() never reads expect — it measures hops. It cannot move any figure the probe prints, and refusing hard-stops a corpus whose frozen question set may not be edited. Kept because a golden set naming a document the index does not hold is broken for make eval, which does read it, and measuring a release precondition against an unchecked question set is not worth the saved minute — but the refusal now says which of its lines move the count and which do not, rather than claiming they all do.

The edge census — a .get default that would have hidden its own defect (20260804 11:25)

MEDIUM — the first draft of the text renderer used report.edges.get(kind, 0), which is exactly the failure class this feature exists to prevent. The whole point of edge_census is that a kind missing from the dict must never read the same as a kind genuinely at zero. _render's first draft indexed the dict defensively, so if edge_census ever regressed to drop a kind — a renamed key, a kind skipped by mistake — the printed table would still show kind 0, silently correct-looking, while the JSON output (built with dict(self.edges), no default) would be missing the key outright. Same bug, two output formats disagreeing about whether it happened. Caught before committing, by asking what a reviewer would ask: "would this line notice its own input being wrong?" Fixed to report.edges[kind] — direct indexing, so a dropped kind crashes loudly in text output too, matching JSON. The instinct to make a formatter defensive is usually right and was wrong here: this formatter's job is to report a fact edge_census promises to supply completely, and a default that papers over the promise being broken is worse than a crash — the same lesson docs/RETROSPECTIVES.md's reachable-ceiling probe retro already drew about the probe's inputs, recurring one layer up, in the probe's own output code.

HIGH — the first edge_census design counted every hub bucket, so co-located and shared-tag could never report zero on a corpus with documents in it, which is exactly the case the feature exists to surface. Found by an independent adversarial-review pass (a fresh agent given the diff and the requirement, not the implementation reasoning), not by the author: a directory holding one document, a tag on one document — plans/20260803_2239-corpus-probe-run.md itself calls this shape "most dirs connect nothing" — still contributed its member count to the sum, because sum(len(members) for members in hub.values()) does not care how large each bucket is. A corpus with real documents and zero shared structure would have printed a large positive co-located, the opposite of what the census exists to show. Fixed by excluding buckets of size one (_spoke_count, tools/reachable_ceiling_probe.py) — a bucket with nothing else in it derives no edge. The reviewer also showed the first reconciliation test could not have caught this: its "independent" expectation was sum(len(members)) over the same kind of total (document count, tag-row count), which is invariant to which bucket each item lands in — a grouping bug and a correct grouping produce the same sum. The fixed test computes its own per-bucket sizes from the raw tables and applies the same size-two-or-more filter, so a fixture now needs at least one real multi-member bucket and at least one genuine singleton bucket (docs/alone/solo.md, deliberately alone) to mean anything.

HIGH — the first reconciliation fixture could not have caught a wrong parent-child formula, twice. 900 words replaced an initial 240: at 240 words per heading section, structural chunking produced exactly one chunk per heading_path, so every groups[a] * groups[b] term was 1 * 1, and a flat += 1 per group-pair passed unnoticed. Raising every group to 900 words moved every group to size 2, not different sizes — and 2*2 == 2+2, so the next plausible wrong formula (groups[a] + groups[b]) also passed unnoticed; the adversarial review caught this one, mutating the multiplication to addition and finding the test still green. Fixed with three unequal group sizes (2, 3, 5 chunks, from 800/1300/2000 words against max_tokens = 510) chosen so no pairwise sum equals the corresponding product. The general form, twice-demonstrated: a fixture built to be "nonzero" is not a fixture built to distinguish the correct formula from the plausible wrong ones near it, and "I already raised the fixture size once" is not evidence the new size clears every nearby coincidence — each candidate wrong formula has to be checked against the actual numbers chosen, not assumed defeated by "bigger."

MEDIUM — the first authored reconciliation counted links rows; edge_census counts unique pairs. graph.authored is a set per document, so two rels between the same two documents (both legal — pnk link refuses only an identical (target, rel) pair, not a second relation between the same target) still count as one edge. The first test's "independent" expectation filtered and counted rows, which happened to agree only because the original fixture had no document pair linked twice. Fixed by comparing unique frozenset({src, dst}) pairs instead of rows, and the fixture now deliberately links one pair twice with different rels to make the distinction observable.

Common thread across all three: an adversarial reviewer with no stake in the implementation asked "would this test's expectation still be right if the code grouped things differently, or if the same edge were recorded twice?" — a question the author, having just written the grouping code being tested, did not think to ask about their own logic.

The docs site — 20260804 11:55

Building a rendered site over docs/ was worth it for one reason before any of the styling: the strict build is a link checker nobody had run over these files. Its first pass reported 31 failures, and they split into two classes worth keeping apart.

Twenty-three were links that leave docs/ — ../plans/…, ../CLAUDE.md, ../tools/record_claude_fixtures.py. Every one of them is correct on GitHub and dangles on the site, because the site only has pages for docs/. These were never wrong; they became wrong the moment a second renderer existed. All are now absolute github.com URLs, which work on both.

Eight were heading anchors, and the fix was not the obvious one. Every failure was a heading containing an em dash or an angle-bracketed code span, and the tempting repair — rewrite the eight links to whatever MkDocs generated — would have fixed the site by breaking the copy people already read in the repo. The three algorithms disagree:

Heading	GitHub	Python-Markdown	pymdownx
## 2.1 The manifest — pinakes.toml	21-the-manifest--pinakestoml	…-manifest-pinakestoml	✓
# The sidecar — <file>.pnk.yaml	the-sidecar--filepnkyaml	✗	the-sidecar--pnkyaml

Python-Markdown collapses the whitespace the dropped em dash leaves behind; pymdownx keeps it but strips anything angle-bracketed as an HTML tag, where GitHub slugs the escaped heading and keeps file. mkdocs_hooks.py implements GitHub's algorithm instead, so the site matches the docs rather than the docs being edited to match the site. The general rule: when a doc is rendered in two places, the renderer that arrived second adapts. Anchors are a published interface.

Two headings carry an angle-bracketed code span today, which is why this is a hook and not two edits — nothing stops a third, and the next one would fail a build with no obvious cause.

Not done, deliberately: PDFScout's 01-getting-started/-style directory tree. docs/ filenames are read by tools/fragments.py, tools/status_header_gate.py, tools/shared_file_overlap.py, tests/test_verification.py, CI and ~150 source comments. The numbered chapters a reader sees come from mkdocs.yml's nav labels and javascripts/section-numbering.js; the directory layout contributes nothing to them.

Renaming the repository broke PyPI trusted publishing — 20260804 12:40

Renaming lucagattoni/Pinakes → lucagattoni/pinakes was checked against everything inside the repo — 64 URLs, the docs site, CI, the gates — and all of it passed. What it broke was outside the repo: PyPI's trusted publisher matches on the exact repository name, and it is registered on pypi.org, where no gate here can see it.

So v0.9.0 tagged, built, smoke-tested, and was refused at the upload:

invalid-publisher — valid token, but no corresponding publisher
repository: "lucagattoni/pinakes"      # the token's claim, post-rename
                                       # the publisher still says "Pinakes"

Nothing was uploaded, which is the only reason this was recoverable: PyPI never allows re-uploading a version, so had the rename landed between two files of the same upload the version would have been burned and 0.9.0 would have had to be skipped. Verified against https://pypi.org/simple/pinakes/ rather than the JSON API, which is CDN-cached and has read as "missing" for a correct upload before (20260729).

The durable lesson: a rename's blast radius is every system that identifies this repo by name, and most of them are not in the repo. Before renaming, enumerate the external identifiers — trusted-publishing publishers, deploy keys and webhooks, any CI in another repo that clones this one, badge and package-index metadata — because a repo-wide grep proves only that the inside is consistent. GitHub's redirect for the old URL is what makes the inside look fine and hides this.

Ordering that would have avoided the outage entirely: update the external publisher first, then rename, then tag. The publisher can name a repository that does not exist yet, so there is no window where neither name works — and the reverse order guarantees one.

Interrupted sync — a TZ test that used the fixture helper would have proven nothing (20260804 12:46)

HIGH — the first draft of the UTC-timestamp regression test called test_sync.py's own run() helper, which hardcodes now="20260725 16:00" for every other test in the file specifically to bypass the real clock. sync.py:709's stamp = now or datetime.now(UTC)... only reaches the real-clock branch when the caller passes now=None — and run(kb, ...) never does, because that fixed string is what makes 60-odd other tests in test_sync.py deterministic. A TZ test built on top of run() would set TZ, call run(kb), read back meta['built_at'], and find it exactly "20260725 16:00" regardless of which clock sync.py actually used — passing identically whether the site under test read datetime.now() or datetime.now(UTC), because neither ever ran. Caught before committing by asking the same question the increment's own instructions insist on for the mutation pass: does the assertion distinguish the fixed from the broken code, or only look like it does? Fixed by calling sync() directly with no now= override in both TZ tests (test_a_real_sync_stamps_utc_not_local_under_a_non_utc_timezone, test_estimate_only_stamps_utc_not_local_under_a_non_utc_timezone — the second reaches its own independent clock at sync.py's _estimate_only, unaffected by the outer now= either way, so it needed no such change but is named here for the record) — and mutation-verified afterward by reverting each site to datetime.now() and confirming the corresponding test failed by roughly the test's chosen offset (Pacific/Kiritimati, UTC+14), not merely failing.

The general rule this confirms rather than discovers: a test helper that fixes an input to make most tests deterministic is exactly the place a new test targeting that specific input must not reuse the helper — the fixture built to remove non-determinism from one property will just as readily remove the property a new test exists to observe. docs/RETROSPECTIVES.md's own advice on claiming a test is mutation-verified — run the mutant, don't just trust the assertion reads correctly — is what caught this one before the mutation pass would have had to.

The progress printer's closing newline assumed the loop always reaches its own end (20260804 13:07)

MEDIUM — an independent adversarial review found that cli._progress_printer's "finished" branch (done >= total) is the only place the printer ever emits its closing newline, and _run's loop (sync.py) does not always reach done == total. A [budget] cap or any early exit stops the loop partway through, so the last progress(done, total) call for that run has done < total, and the printer's \r-prefixed line is left open — no trailing newline — for whatever prints next (print_sync_report, or an error message on an unhandled exception) to land on. Confirmed live: a progress line followed immediately by report text on the same terminal row. No test in the original commit exercised done < total as a run's last call — test_progress_printer_throttles_... always ended at done == total, and the progress-callback sync test never triggered an early stop.

Fixed by splitting _progress_printer() into (progress, finish): progress behaves as before, but also tracks whether a line is open (dirty); finish closes it with one newline if so, and is a no-op otherwise. run_sync calls finish unconditionally in a finally around the sync() call, so it closes the line whether the run finished normally, stopped on a budget cap, or sync() raised. Mutation-verified: made finish() unconditionally clear dirty without printing; test_progress_printer_finish_closes_a_line_an_early_stop_left_open caught it (expected "\n", got "").

The general shape of the miss: a "does this print the right thing" test that only drives the happy path (the loop's last call always being its final call) cannot see a defect that only exists on an early-exit path, because the assertion and the code under test share the same unstated assumption — "the loop reaches done == total" was never itself questioned, only how the printer behaves once it does.

A green ./check.sh only proves the worktree's own venv is green (20260804 13:30)

MEDIUM — test_estimate_only_stamps_utc_not_local_under_a_non_utc_timezone shipped with no pinakes[pdf] skip marker, and ./check.sh was green anyway, because this worktree had [pdf] installed. The test writes a real baseline-1p.pdf and calls page_count on it for real (its own docstring already said so), which needs pypdfium2. The planner's worktree did not have the extra installed and hit AttributeError: module 'pypdfium2' has no attribute 'PdfDocument' at merge time — same commit, same script, different venv. CI's check job runs a three-leg matrix over [light], [light,pdf] and [light,pdf,claude] specifically because core stays torch- and pypdfium2-free by design (CLAUDE.md § Tooling); this would have gone red on the [light] leg after merge, which the merge-time worktree-mismatch is what actually caught here.

The rule this earns: a green ./check.sh run proves the worktree's own dependency set is green, not the matrix — pytest silently skips whatever the installed extras cannot exercise rather than failing loudly, so a test that forgot its @pytest.mark.skipif(not pdf_extraction_runnable(), ...) marker doesn't skip or fail locally when the extra happens to be present; it just runs, passes, and says nothing about the leg where it can't. Before landing any test that touches a PDF fixture, the claude extractor, or a real embedding backend (sentence-transformers/fastembed), check which pdf_runnable()/pdf_extraction_runnable()/ paid_runnable() predicate (tests/conftest.py) it needs and mark it — and separately, run at least the affected file with the relevant extra uninstalled (uv sync --extra light --frozen, run, then uv sync --extra light --extra pdf --extra claude --frozen to restore), because that is the only way ./check.sh passing locally actually predicts the [light] CI leg rather than just restating the worktree it ran in.

G3 — the node model and the edge set (20260804 16:30)

HIGH — a hierarchy derivation that was quadratic in a document's chunk count. parent-child was derived by testing child.heading_path.startswith(parent.heading_path + " > ") over every chunk pair within a document. Measured on one document: 2 000 chunks 0.23 s, 4 000 chunks 0.85 s, 8 000 chunks 3.32 s — so a single 32 000-chunk document (one long PDF) would have spent ~50 s deriving, on a path pnk sync runs from three git hooks. Replaced by grouping chunks by heading path and having each path look up its own ancestors (" > ".join(segments[:d])), which is the same relation and is linear in chunks: 32 000 chunks now 0.59 s. The corpus that would have exposed this is the one that cannot: the RFC realism corpus has an empty heading_path on every chunk, so its 106 806 chunks derived zero hierarchy edges and cost nothing. A performance defect invisible on the only corpus at scale.

HIGH — a mutant that nothing caught, in a filter that looked redundant. authored_pairs filters links on src_kb_id = ? AND dst_kb_id = ? and then joins both ends to doc nodes. Changing the AND to an OR failed no test, because the join already drops a foreign document ULID — a foreign document has no local doc node. That reasoning is wrong in exactly one case, and it is a case that happens: fork a KB — copy the directory, mint a new [kb] id, and every document keeps its permanent ULID. A reverse scan of the fork then writes (fork_kb, D, local_kb, E) where D is also one of our documents, and the OR reads it as "our D cites E" — an edge nobody authored here. The lesson generalises: a filter that a second filter appears to make redundant is only redundant under an assumption, and the assumption is the thing to test. Pinned by test_a_forked_kb_sharing_a_document_ulid_does_not_forge_a_local_authored_edge, which builds two real KBs rather than inserting the row.

MEDIUM — the deriver was cross-checked against the instrument the go decision was measured on, and this was worth more than any single test. tools/reachable_ceiling_probe.py derives the same relations in memory, written independently. Comparing the two censuses on tests/demo-kb (test_the_stored_edge_set_agrees_with_the_probe_the_decision_was_taken_on) caught two mutants no targeted test did, and the RFC corpus reproduced the go decision's drop table exactly — sibling 106 506, shared-tag 643, co-located 262. A second implementation of the same spec is a cheaper oracle than a third round of hand-written assertions, and it answers a question no assertion can: did G3 build the graph G2 measured, or a different plausible one.

MEDIUM — the plan's orientation rule, read literally, makes every hub unreachable. G3's spec says the provider queries "src = ? OR dst = ? for those kinds and src = ? for hub kinds". A hub spoke is stored hub-first, so src = ? answers "who is in me" — and a member asking "what am I in" needs dst = ?. Read literally, no member could ever enter a hub, and co-located/shared-tag are the two kinds the go decision measured carrying all nine liftable questions. The sentence's real content is the symmetric half: a src-only read of a symmetric kind silently drops half of every relation. Built as three explicit functions — peers(), members(), hubs() — so the two halves of a hub kind cannot be confused for one query, and pinned by test_a_hub_is_entered_from_a_member_and_expanded_from_the_hub.

HIGH — a kind selection that could be silently dropped, in the function that preaches against it. peers()/neighbours() took local_kb: str | None = None and skipped authored when it was absent — the same "confident, wrong, smaller answer" its own docstring warns about for a src-only read, from the other side. G5's gate runs with and without authored edges; a caller who forgot the keyword would have got the "without" arm believing it ran the "with" one, and lost the highest-trust edge class with nothing printed. Now a TraversalError naming select_kinds(drop=["authored"]) as the way to mean it. select_kinds() refusing an unknown name was designed against exactly this and did not cover it — the refusal guarded the name and left the ingredient.

MEDIUM — every corpus that certifies this increment has two of the six kinds at zero. parent-child has derived zero edges in every measurement ever taken here — one-level headings in both committed corpora, an empty heading_path throughout the RFC corpus — and no committed sidecar carries a tags: key, so shared-tag is exercised only by synthetic fixtures. The cross-check against the probe therefore passed with _hierarchy_edges deleted, because its non-vacuity guard was a total over all kinds. Fixed by asserting each compared kind is non-zero and naming the two this corpus cannot exercise, so a kind that silently stops deriving fails rather than reads as agreement. A "total > 0" guard on a per-item comparison is not a non-vacuity guard — it is one item's evidence spread over all of them.

MEDIUM — parent-child is materialised pairwise, and its row count is the product of two sections. APPROACH §3 keeps hierarchy direct — "the one relation that stays direct" — so an ancestor heading of a chunks and a descendant of d chunks is a·d rows. Measured at max_tokens=400 on plausible document shapes: 5.8×, 16.3× and 53.5× the chunk count. Extrapolated to 300 documents of the worst shape that is ~10 M rows. Not changed here — hubbing it contradicts APPROACH §3 and restricting it to the immediate parent narrows the relation the go decision's probe measured — but pinned by a test that asserts the arity, and reported to the planner as a spec question rather than absorbed as an implementation choice.

LOW — a duplicated tag inflated a hub's member count where nothing could see it. derive collects edges into a set, so tags: [t, t] produced one spoke — but _tag_buckets appended the document twice, and the < 2 minting rule counts the bucket. A single document repeating one tag therefore minted a hub with one spoke and a divisor of 2. The test named test_a_duplicate_tag_in_one_sidecar_is_one_spoke passed with the deduplication removed, because the set downstream hid the mechanism it named. Deduplicated where the length is decided, and pinned by the single-document case that has no set to hide behind.

LOW — a hub with one member is derived state that connects nothing. A directory holding one document, a tag on one document, a heading with one chunk: expanding it returns only the node that reached it. The spec only says degree-zero hubs are reaped, which full re-derivation gives for free. Degree-one hubs are minted at zero benefit — a node, a spoke, and an entry in G6's hub report — so they are skipped, which also makes the census directly comparable to the probe's (_spoke_count counts buckets of two or more). Reachability is unchanged for the channel as APPROACH §4A defines it today; §4B's all-chunk seeding is itself flagged for re-evaluation on the sqlite-vec tier, so the claim is scoped rather than unconditional. The alternative reading is recorded here because it was a choice, not a deduction.

G5 — the parent-child ceiling, measured before the gate ran (20260804 21:05)

The measurement the arity decision required, and the corpus that could supply it was neither of the two anyone would have reached for. plans/20260804_1844-decision-parent-child-arity.md keeps parent-child transitive and asks for a ceiling "against a corpus whose chunker actually populates heading_path" — because the projection of 5.8×–53.5× the chunk count had never been run against one. Both obvious candidates fail for different reasons, and only the second was known:

corpus	documents	chunks	carry a heading_path	heading depth	parent-child
RFC realism	300	106 806	0	—	0 — structural chunking degraded silently
tests/demo-kb	30	60	60	always 1	0 — every document is flat
this repo's docs/ + plans/	43	2 671	2 671	median 2, max 4	13 232

tests/demo-kb populates heading_path on every chunk and still derives zero hierarchy edges, because a depth-1 path has no ancestor. A corpus can satisfy "the chunker works" and still exercise nothing, which is a second way to get a zero that looks like a measurement — and it is the shape the go decision's own bound warns about, one layer in.

The numbers

Real Markdown, written by hand, with real nesting — 43 of this repository's own documents, indexed into a scratch KB (never committed; docs/ and plans/ are the corpus):

• 4.95 parent-child rows per chunk — 13 232 over 2 671 chunks. Against sibling's 2 628 and in-section's 2 509, hierarchy is 71% of every stored edge, on a corpus where sibling is one row per adjacent chunk. It lands below the 5.8× floor the decision projected, so the projection was pessimistic rather than optimistic.
• Derivation costs nothing. The whole edge set derives in 0.158 s; the hierarchy alone is 0.004 s for those 13 232 rows. G3's ancestor-lookup form is linear in chunks and quadratic only in a document's distinct heading paths (median 11 here, max 76), exactly as it claims. The cost is row count, never wall clock, which is what the decision predicted and is worth stating because it changes which mitigation would ever be needed.
• Index growth is 12.9% — 11.17 MB with the hierarchy against 9.89 MB without, both VACUUMed. On a corpus of this shape the absolute number is 1.2 MB, and it scales with rows, not bytes of text: an index dominated by 384-dimensional embeddings (the RFC corpus is 265 MB for 106 806 chunks) would grow proportionally far less.

The ceiling is not alarming, and the standing risk is unchanged

Extrapolating 4.95 rows/chunk to the RFC corpus's 106 806 chunks gives ~529 000 hierarchy rows against its 107 802 total today — a five-fold graph, derived in well under a second. That is a number, not a problem, and it does not license switching to the immediate-parent variant: the decision is explicit that the variant is the arm to measure if the ceiling proves alarming, never the default to switch to first.

What is still unmeasured is the tail. 4.95 is a mean over documents whose median depth is 2. The decision's standing risk — "a corpus with deep heading nesting and large sections could make parent-child the dominant kind" — is about a shape none of these three corpora has: deep nesting and many chunks per section, where the row count is the product. This corpus's worst document carries 76 distinct heading paths and its arity stays modest because its sections are short. Nothing here refutes that risk; it bounds the ordinary case and leaves the tail where the decision left it.

The tail, measured (20260804 22:39) — and it is alarming

The paragraph above left the standing risk as prose. It is now a number. A purpose-built worst-shape corpus — six documents, heading depth 4, every heading path carrying ~26 chunks, which is the a·d product the risk names — was generated, synced with the same real backend, and measured the same way:

	this repo's docs/ + plans/	worst-shape corpus
chunks	2 671	2 483
heading depth, median / max	2 / 4	3 / 4
chunks per heading path, median	short sections	26
parent-child rows	13 232	132 630
rows per chunk	4.95	53.42
share of every stored edge	71%	94.7%
index growth	+12.9% (1.2 MB)	+113.4% (13.3 MB)
derivation	0.004 s (hierarchy)	0.84 s (140 079 edges, every kind)

53.42 lands at the very top of the decision's projected 5.8×–53.5× band, so the projection was accurate at both ends rather than pessimistic: 4.95 sits below its floor for ordinary prose, 53.4 reaches its ceiling for the shape it warned about. The index more than doubles. Derivation stays cheap — 140 079 edges in 0.84 s — which confirms the decision's own prediction that the cost is row count, never wall clock, and therefore that no mitigation aimed at derivation time would help.

What this corpus is, and is not. Synthetic and deliberately adversarial: generated Markdown with uniform nesting and uniform section length, built to make the product as large as plausible rather than to resemble anyone's notes. It is not evidence that real corpora do this — neither real corpus measured above comes close. It is evidence that the shape is reachable without anything exotic, because depth 4 with long sections is an ordinary specification or manual.

No variant is switched to here, on purpose. plans/20260804_1844-decision-parent-child-arity.md is explicit that if the ceiling proves alarming the immediate-parent form is the arm to measure, never the default to change first — and --drop parent-child is already a reported leg of G5's matrix. This is the input to that decision; the decision is the planner's.

G5 — the expansion channel (20260804 21:35)

HIGH — a ranking rule that would have made depth 2 unreachable, and it read as the careful choice. The channel's output is ordered and then cut at candidates_per_source. The first implementation ranked (distance, -cosine, …), on the reasoning that a two-hop chunk should not outrank a one-hop one on similarity alone — which sounds like the conservative reading of APPROACH §4A's "score expanded chunks by edge weight and link distance". It is not conservative, it is silently disabling half the feature: with distance as the primary key, every one-hop chunk precedes every two-hop one, so on any corpus where one hop already fills the cut, depth 2 contributes nothing to the output at all. The channel would have been depth-1 wearing a depth-2 budget — and the reachability ceiling that unblocked this increment was measured at two logical hops, so the gate would have been measuring something the precondition never covered. Now (-cosine, distance, …): cosine ranks, distance breaks the ties cosine cannot, and test_a_two_hop_chunk_outranks_a_one_hop_one_when_the_query_says_so fails if the order comes back.

HIGH — every tiebreak resolved to a surrogate id, and it looked stable because of another module's ORDER BY. Fan-out, frontier order and the final ranking all broke ties on nodes.id. Those ids are deterministic today — but only because edges.derive enumerates documents ORDER BY path — so the channel's answers depended on an invariant of a different module, unstated in both. That is G1's defect in a new place: _hydrate's unordered WHERE c.id IN (…) was stable in practice too, until a rebuild moved one golden-set question. Every tiebreak here now resolves to (documents.path, chunks.ordinal), the total order G1 gave the rest of the pipeline. The lesson is not "avoid surrogate ids" — it is that "this is deterministic" is a claim about the code that computes it, and if that code is somewhere else, the determinism is an assumption rather than a property.

HIGH — a fixture that was not the shape its own docstring described, in the tests written to catch exactly that. The membership-exclusion fixture built one document of six flat sections, so that ordinal 0 and ordinal 4 would be reachable only through their document's membership edge. Its body was f"## Section {i}\n\nword{i} " * 30 — which repeats the heading thirty times, and produced 180 chunks nested under Section 0 > Section 5. A parent-child hierarchy, inside a fixture whose entire purpose was to have none, in a file whose docstring names "an assertion satisfied by something other than the property it names" as the failure it is written against. It was invisible until an unrelated ordering change made the walk reach one chunk more. Two things followed: dropping the # Title was not enough either — the chunker then reads the first ## as the root and derives Section 0 > Section n, the same defect one heading level down — and a fixture that a test's meaning depends on now asserts its own shape (_assert_flat_sections: six chunks, every heading path depth 1, zero parent-child edges). A fixture is an assertion.

MEDIUM — a mutant that survived eighteen others, in the half of a rule the other half hides. The membership exclusion is two filters: a document never passes through to itself, and a root's own document never contributes member chunks at any depth. Deleting the second left the whole suite green, because at hop 1 the source is the root's document, so the first already covers it — and no fixture ever reached a root document from somewhere else. The shape that does is A —authored→ B —authored→ {A, C} at adjacent_k=1: at hop 2 the source is B, the first filter does not apply, and without the second, A takes the only slot to re-contribute chunks the query already had. Two rules that agree on every case you have built are one rule until you build the case that separates them.

LOW — a throwaway mutation harness produced two runs of contradictory results. Mutating a source file, running pytest and restoring in a loop reported extra failures in tests unrelated to the mutated file, twice, while the machine was also running a two-hour re-index and an eval matrix. Clearing __pycache__ between mutants and running each mutant on a quiet machine gave seven clean, reproducible results. The mechanism was not pinned down — a stale bytecode cache after a same-second restore and a subprocess that failed to spawn under load are both consistent with what was seen. Recorded because the reaction is the reusable part: a mutation result that implicates a file the mutant did not touch is a result about the harness, and re-running it on a quiet machine costs a minute.

G5 — a root spends a fan-out slot it is then discarded from (20260804 22:30)

HIGH — the fourth review round, on a rule the third round had already written down. Round three added a filter to _offer_chunks dropping candidates already found this hop or already emitted by an earlier one, with the reasoning stated in the comment: "a slot spent on it adds nothing, and dropping it before the cut is the same rule the membership exclusion applies one level up". The rule is right. It was applied to two of the three categories it covers.

A root reaching found is discarded twice over. _accept skips it before emitting, and run seeds self._expanded with the roots so it never joins the frontier either — so a root contributes neither a row nor a hop. Yet it had already taken one of the adjacent_k slots on the way, and the neighbours of a fused top-k chunk are very often other fused top-k chunks: sibling connects adjacent ordinals, and adjacent ordinals of a chunk the query matched are exactly what the vector stage also ranked highly.

Measured on graded_neighbour at the shipped default adjacent_k = 8: 4 candidates returned as built, 10 with roots dropped before the cut — and all six of the new ones are chunks fusion had not found, which is the only thing the channel exists to contribute.

Why it survived three rounds. Every existing assertion about roots was set-level — "neither root may be emitted", "expansion still runs" — and a set-level assertion cannot tell a slot spent from a slot saved. Both are satisfied by the defective walk. The test that catches it counts instead: adjacent_k = 1, two candidates, one of them a root, and the single slot must reach the non-root. With the filter removed the walk returns [], which is the failure mode named in the assertion's own message.

What the fix is worth, and where that cannot be seen

tests/demo-kb moved zero questions across all three gate legs — off, expand, expand-no-authored — with aggregates identical to four decimal places. That is not evidence the fix is inert: demo-kb's documents are about two chunks each, so adjacent_k = 8 never saturates and a root never displaces anything. A corpus can be incapable of exercising a change, and reporting "no movement" from one is reporting the corpus, not the code. The RFC realism corpus — 300 documents, 106 806 chunks — is where the fan-out cut actually binds, and it is the corpus the gate runs on.

The backstop that now survives mutation, on purpose

_accept's if node in self._roots: continue is unreachable once _offer_chunks filters. It stays: "a root is never emitted" is an invariant of the emit point, and a later caller adding a second way into found should not be able to break it from a distance. The honest consequence is recorded in the docstring and in the mutation table — deleting that line fails no test, while deleting the _offer_chunks filter fails exactly one. A backstop whose mutant survives is fine; a backstop silently counted as the enforcement is not, which is the same assertion-satisfied-by-something-else failure in its bookkeeping form.

G5 — the gate, run: expand ships off (20260804 22:52)

HIGH — the increment's deliverable is a negative measurement, and it is a clean one. On the RFC realism corpus, at G5's own HEAD, against a schema-3 index rebuilt for this run:

leg	multi-hop	improved	regressed	p
off	7/20	—	—	—
expand, drop authored	4/20	0	3	1.0000
expand, all kinds	4/20	0	3	1.0000

licensing p = 1.0000 (max, the more conservative of the two). expand defaults off, and tools/graph_gate.py exits 1. Clause 1 fails in both runs; clause 2 fails as well — by_kind[multi-hop] 0.350 → 0.200 against a 0.02 tolerance — and clause 4 fails on recall_at_k for the same movement.

Lost: content-disposition-http-takeover (hit at rank 1 → not found at all), http-rate-limit-status-code-provenance (rank 4 → miss), sip-invite-2xx-retransmission-defect (rank 2 → miss). Nothing was lifted.

reachable ≠ retrievable — the finding, and the plan predicted its shape

The reachability probe found 9 of the failing multi-hop questions reachable within two logical hops without authored edges, which is what unblocked the graph release. The retrieval instrument lifts none of them, and the channel's extra candidates displace three answers two-list fusion already had. The plan drew this distinction itself — "a ceiling gauge cannot rank, and an argument cannot measure" — and G5's whole existence is the reason it could be checked rather than assumed. A reachability precondition is necessary and nowhere near sufficient, and the gap between the two is not a small correction: it is 9 questions against 0.

The go decision also anticipated the outcome's meaning: "If expand did not pass, the finding is 'graph structure does not help this corpus', and the response is a corpus or a different channel design, never an escalation to a more expensive one." Nothing here licenses PPR.

Two things that make the number narrower than it looks, both stated rather than worked around

Clause 3 passes vacuously, and clause 4 nearly so. The RFC corpus has [retrieval.confidence] commented out, so every question scores confidence: unknown; neither the confidence-lost nor the newly-found-at-low term can be non-zero there. The decomposition clause 3 exists for is exercised only by the synthetic artifacts in tests/test_graph_channel.py — which is precisely why the plan insisted the gate be driven by synthetic fixtures as well as by the corpus. A gate whose only fixture is the real corpus can only be tested in whichever direction that corpus happens to point, and here two of its four clauses would never have fired.

The --drop parent-child arm is inert on the gating corpus. No chunk in the RFC corpus carries a heading_path, so parent-child and in-section derive zero edges and dropping them changes nothing by construction. The arm the arity decision added cannot say anything here. Its cost was measured separately (see the ceiling fragment); its retrieval value remains unmeasured, and the corpus that could measure it does not exist yet.

The --drop sibling arm answers its question, and the answer is "neither"

The go decision added this arm to ask, with the instrument that measures retrieval quality rather than a reachability ceiling, whether 99.2% of the graph's mass earns its place. On this corpus sibling is 106 506 of the 107 411 non-transit structural edges, and dropping it produces exactly the same 4/20, the same three regressions, and the same p = 1.0000 — one question's rank moves, and it is a miss either way.

So sibling neither helps nor hurts. It is 99.2% of the stored graph and it is inert in both gauges: the reachability probe already found removing it cost nothing, and the retrieval instrument now agrees. Two independent measurements, and the harm the channel does comes from somewhere else entirely — the document-level path, membership transit into co-located (262 edges) and shared-tag (643) hubs, which pull whole documents' chunks into the fusion.

With the caveat that makes it a narrower claim than it looks: every chunk here has an empty heading_path, so a "sibling" in this corpus is an adjacent arbitrary size-slice, not an adjacent section. The arm has measured the value of size-slice adjacency, which is what this corpus's broken structural chunking produced — not the value of sibling as designed. On a corpus whose chunker works the question is still open.

Five legs land on the same number; the sixth does not, and I had already written that they all did

The full matrix, off at 7/20 (recall@k 0.3500, MRR 0.421):

leg	multi-hop	improved	regressed	p	ms/query
off	7/20	—	—	—	2012
expand	4/20	0	3	1.0000	2051
expand-no-authored	4/20	0	3	1.0000	2106
expand-no-sibling	4/20	0	3	1.0000	2067
expand-no-parent-child	4/20	0	3	1.0000	2028
expand-no-link-distance	4/20	0	3	1.0000	2024
expand-in-degree	6/20	1	2	0.8750	2237

MEDIUM — the process finding, and it is the file's own failure class caught in the act. With five of the six legs written I recorded that "every leg lands on the same number, which is what makes the result robust" — a claim about six legs asserted from five, while the sixth was still running. The sixth then came back different: in-degree salience is the only configuration that lifts anything (imap-utf8-two-strategies), and the only one that regresses two rather than three. The claim was wrong within minutes of being written, in a fragment whose subject is an assertion satisfied by something other than the property it names. A generalisation over N runs written while N−1 have finished is not a measurement, it is a prediction wearing a measurement's clothes. The correction is recorded rather than quietly overwritten, because the tempting fix — waiting and writing the true sentence — would have hidden how easy it was to write the false one.

What the sixth leg does and does not license. expand-in-degree at 1 improved / 2 regressed is p = 0.8750: nowhere near the gate, and still a net loss of one question against off. It is reported, never gated — three variables against one threshold is not a decision procedure — and noticing that it is the best-performing leg after seeing the numbers is exactly the exploratory fitting the pre-commitment forbids. It is a direction for a future measurement on a corpus that can carry one, not a result.

Latency, the other exit criterion. off 2012 ms/query against expand 2051 — 1.02×, so the channel costs about 2% on a 106 806-chunk index and the "slow at query time" risk did not materialise. In-degree salience is the expensive leg at 2237 ms (1.11×), which is the one place the matrix's timing separates the configurations at all.

The with/without-authored split is stronger here than on tests/demo-kb

All 391 of the corpus's authored links are intra-KB — every to: names the corpus's own KB ULID — so unlike tests/demo-kb, where only 12 of 16 survive the cross-KB inertness rule, the with-authored leg has every one in play. They still move only two questions' ranks and flip no outcome, and both gated runs regress the identical three questions. The anti-circularity guard had nothing to catch, because there was no win to be circular about.

The pre-commitment held

A result short of the table ships the channel off, with counts and p-value recorded, untuned. Nothing was tuned, no weight moved, no threshold was revisited after seeing the number. The authored weight's measured at G5 marker is discharged by this run as "measured, and it changed no outcome" rather than by a fitted value.

G6 — Edge-hub reporting (20260805 06:20)

MEDIUM — the enumeration order this check's own sort relied on was implicit. The first cut enumerated hub node ids with SELECT DISTINCT src FROM edges WHERE kind = ?, no ORDER BY — the only query touching edges in this codebase without one; every read in graph/edges.py orders explicitly, and that file's own docstring argues for it. It happened to come back in ascending src order today (a covering-index scan on edges_src(src, kind)), which is exactly the "mint order" the first test's fixture needed to differ from degree order to prove the .sort() was doing real work. That property held by accident of a query plan, not by anything this function asserted. Fixed with an explicit ORDER BY src. Found by an adversarial review agent, not by any test — no test could have caught it, since the property under test (the sort) still worked; only reading the query against the file's own convention surfaced it.

MEDIUM — one of the three hub kinds shipped with zero assertions on its printed text. _hub_label has three branches — tag, dir, heading — and the landing commit tested tag (the sort-order fixture) and heading (the human-actionable-label fixture) but never dir. co-located appeared in a docstring only, explaining why the other fixtures were built to avoid triggering it — which reads, on a second look, exactly like the tests were routed around covering it rather than covering it. test_a_directory_hub_is_named_by_its_kb_root_relative_path closes it.

LOW, folded into the fix above — a degree tie had no assertion in either direction. top.sort(key= lambda item: item[1], reverse=True) is a stable sort, so two hubs at equal degree kept whatever order they arrived in — which is to say, the same implicit query-plan order the first finding flags, one layer further in. Given a real tie the printed order was accidental twice over. Fixed by sorting on (-degree, kind, key) explicitly, and test_a_degree_tie_breaks_deterministically_and_ the_rest_are_counted builds four hubs tied at degree 2 whose mint order is the reverse of their correct printed order — reverting the tiebreak makes the mutant print d, c, b instead of a, b, c, which is the sharpest test in this increment: it fails on a reverse=True-only sort that every other test here would still pass.

LOW, self-inflicted — a vacuous assertion, written by the implementer, in this same increment. While extending an existing test's block with a new line, an assertion checking for a string that appears nowhere in src/ — "unchecked until the links release" not in detail — was added twice (once by the edit that introduced the increment's other tests, once copied into a third test written later in the same session). It always passed, proved nothing, and is exactly the failure class this project's own CLAUDE.md names: "an assertion satisfied by something other than the property it names." Caught by cross-referencing the diff against src/ during this same retrospective pass (grep -rn "unchecked until the links release" src/ returns nothing), not by any tool — nothing in check.sh can distinguish a true-but-empty assertion from a load-bearing one. Removed.

Read together: three of these four findings trace to the same root — an implicit order (query plan, sort stability) standing in for an explicit one, discovered because it happened to agree with what a fixture needed. G3's own docstrings warn about exactly this shape for a src-only read of a symmetric edge kind ("a confident, wrong, smaller answer"); this increment reproduced a milder version of it one level up, in the read that reports G3's own structure back to a human.

The [light] backend error — a fixed test that only looked environment-independent (20260805 07:41)

MEDIUM — the existing test_a_missing_extra_names_the_install_command was silently coupled to which extras happen to be installed in the checkout running it, and this dev checkout already has fastembed (a transitive dependency of some other extra) even without [light] explicitly requested. Once BackendMissingError started naming an installed alternative, that test's monkeypatch.setattr(builtins, "__import__", refuse) — which only blocks sentence_transformers — left fastembed genuinely importable, so load_backend picked it up as the alternative and the old assertion ('uv add "pinakes[st]"' in remedy) started failing for the right reason: the new code path executing, not a bug. A version of this test that had merely added the new assertions without also forcing "nothing else is installed" would have been true by luck of this machine's site-packages, not by construction, and would flip on a bare CI leg or a machine with no fastembed at all. Fixed by monkeypatching importlib.util.find_spec directly in both the "no alternative" and "alternative present" tests, so each names its precondition instead of inheriting whatever the environment happens to have — the same discipline docs/RETROSPECTIVES.md already names for tests that read like they exercise a real-clock or real-install branch but route around it.

Confirmed by the mutation pass: forcing _installed_alternative's find_spec check to always report "installed" broke test_a_missing_extra_names_the_install_command specifically (asserting alternative is None), and reverting _import to drop the detected alternative broke both alternative-path tests specifically (asserting alternative == FASTEMBED) — each mutation failed the test that names the property it broke, not an unrelated one.

The heading-coverage check — two findings, one of them about my own test

HIGH — the open correction's diagnosis was approximately right and precisely wrong, and the difference decides the fix. Item 1 read "the heading grammar is Markdown-shaped; RFC section numbering is not, so nothing matches and the strategy quietly becomes size-based". That describes a regex being tried and failing. What actually happens is chunk.py:131:

blocks = _markdown_blocks(text) if kind == "markdown" else _plain_blocks(text)

_markdown_blocks is never called for a .txt file, and _plain_blocks sets heading_path=None unconditionally. Nothing failed to match because nothing was tried. The consequence for the fix is not cosmetic: tightening or extending a regex would have changed nothing, and the real change is adding heading detection to a code path that has none. It also bounds the blast radius in the useful direction — tests/demo-kb is Markdown, so a plain-text grammar cannot move the golden set, and "changing chunking needs eval justification" becomes a thing you can prove rather than argue.

A measurement replaced a threshold. Chunking the committed corpora directly — no index, no embeddings, chunk_document called in a loop — gave demo-kb 60/60 and partner-kb 55/55 at 100% against the RFC corpus's 0%. Bimodal, so the predicate can be "zero for this source type" and the check carries no constant anybody had to calibrate. The alternative, a fitted percentage floor, would have needed a corpus to fit against and would have fired on ordinary documents whose opening paragraph precedes their first heading.

MEDIUM — mutation testing refuted one of my own tests within a minute of writing it. I wrote test_heading_coverage_counts_only_active_documents, asserting the state = 'active' filter. M1 deleted the filter and the test stayed green. The reason is that SoftDelete drops a document's chunks as well as flipping its state, so a chunk-counting query has nothing to over-count either way — unlike _links, which counts documents and genuinely needs the filter it records having shipped without (2 of 1 documents linked (200%)).

The test was kept and renamed to what it proves (test_a_removed_documents_chunks_stop_being_counted), with the refutation written into its docstring so nobody re-derives the wrong claim from the old name. The filter stays as defensive consistency with _links, marked as unreachable by this fixture rather than presented as guarded.

This is the file's own recurring failure class caught in the act: an assertion satisfied by something other than the property it names. Three of the four mutants died as intended (the zero-per-type predicate, the two-cause remedy split, the WARN status); the one that survived was the one whose name made the strongest claim. Green proved the test ran; only the mutant proved what it could detect — and for one of five, the answer was "not the thing in its name".

Item 5 — doctor never prints the operator's home directory (20260805 08:06)

LOW — two residual home-directory leaks exist, both correctly out of this increment's scope. An adversarial review confirmed the sweep of src/pinakes/doctor.py is exhaustive against every PinakesError subclass doctor.py forwards, and found no defect in the fix itself (three independent mutations — a no-op _de_homed, a swapped _local tuple order, and a reverted _sidecars call site — each broke exactly the test that should catch it, and nothing else). Two things remain that print an absolute path containing the operator's home directory, both outside the item's stated boundary ("paths outside the KB stay as they are"):

1. _linked_kbs's except OSError as exc: absent.append(f"{linked.name} ({exc.strerror or exc})") — the or exc fallback stringifies a bare OSError, whose default __str__ includes .filename when set. The path involved is a linked KB's resolved location, not manifest.root — legitimately a different KB elsewhere on disk, not this one — so it falls outside "paths outside the KB stay as they are" by the same reasoning that already keeps hf_cache_dir() untouched. Rare: only fires when why_not_a_kb's OSError.strerror is falsy, an edge case its own docstring already calls out as rare (an unreadable parent directory).

2. budget.prices.PricesMissingError(reason=str(exc)) — ships a package-relative path (prices.toml's location inside the installed wheel or an editable checkout), not manifest.root-derived, so _de_homed correctly leaves it alone. In an editable/source install, that path is often literally under the developer's home directory too (e.g. ~/Code/pinakes/src/pinakes/budget/prices.toml) — the same shape of leak as hf_cache_dir(), for the same reason (a real filesystem location worth showing, not KB-derived).

Worth keeping: if "no home directory in pnk doctor output, ever" becomes the actual goal rather than "no home directory via the KB's own location", both of these are where to look next — they were not fixed here because the item's own text draws the boundary at manifest.root, and extending it is a separate decision.

measure_sync_cpu.py measured the launcher, not the work (20260805 17:37)

HIGH — the tool answered its one question with a number that was precisely, confidently wrong, and every test passed. sample_percent ran ps -o %cpu= -p <pid> against the pid it launched. The invocation the tool was written for — and prints in its own --help and changelog fragment — is -- uv run pnk sync ..., which makes uv the measured process and pnk its child. uv waits and burns nothing.

Measured on this repo before the fix, one identical one-core busy loop:

launched as	reported
the busy loop directly	1.0 cores
the same loop behind uv run	0.0 cores

The failure mode is the expensive one: 0.0 cores for a sync saturating a core does not read as a broken tool. It reads as the finding item 6 went looking for — "the loop is not CPU-bound, so multiprocessing buys nothing" — and it would have been quoted into a design decision.

Why the tests could not catch it. All seven ran sys.executable -c <busy loop>: a direct child that does the work itself. The suite covered the sampler, the units, exit-code propagation, empty and non-positive arguments, and the trailing-interval bug — everything except the one process shape the tool exists to be pointed at. Coverage of the code was complete; coverage of the invocation was zero. This is the recurring class named in docs/RETROSPECTIVES.md — an assertion satisfied by something other than the property it names — reached from a new direction: not a weak assertion, but a fixture that was never the real subject.

Fixed by summing %cpu across the root pid and every descendant from a single ps -A snapshot (one snapshot, so a child starting or exiting mid-walk cannot be double-counted or missed), plus a test whose command is a launcher that burns nothing itself.

The new test's upper bound is load-bearing, and mutation proved it. With the tree walk neutered, the launcher still reported 0.1 cores of its own interpreter startup — so assert peak > 0 would have passed the mutant. Asserting > 0.5 fails it. A threshold above "anything a waiting process can produce" and below one core is what makes the assertion name the property; "non-zero" would not have.

Also corrected: %cpu is a decaying average over up to a minute (man ps), not the instantaneous reading the docstring claimed. Right for the steady-state multi-minute loop this measures, but it means peak is the peak of a smoothed series — a low peak is much weaker evidence of an idle machine than a high peak is of a busy one, and the docstring now says so.

Generalisable: when a tool's purpose is to be run one particular way, one test must run it that way. A synthetic fixture chosen for speed silently replaced the subject here, and no amount of assertion strength on the wrong subject would have helped.

Fixture copies carried the developer's own .pinakes/ into the test workspace (20260805 17:49)

MEDIUM — three tests failed on main immediately after two clean merges, on a machine where nothing was wrong with the code, and the failure impersonated the exact defect this repo watches for. CLAUDE.md says a clean auto-merge is not a correct merge; both branches had been green individually; main then failed. Every signal pointed at a bad merge. The cause was a tests/demo-kb/.pinakes/index.db dated 1 Aug — a leftover from a manual pnk sync predating the graph release's schema_version 3 bump — which five shutil.copytree calls copied into the test workspace along with the documents.

The tests were coupled to whether the developer had ever run the tool by hand. CI clones fresh, so .pinakes/ never exists there and the suite is permanently green; a dev box that has exercised the fixture once fails until the directory is removed. .pinakes/ is gitignored, so nothing in the diff, the merge or the branch could have shown it.

The idiom was already in the codebase and applied to four of nine call sites. ignore=shutil.ignore_patterns(".pinakes") was used in test_eval.py, test_search_reproducibility.py and twice in test_partner_kb.py. The other five simply omitted it. A guard applied at some call sites is not a guard — it is a coin flip weighted by which test the copy happens to be in, and no gate could notice the omission because the states it protects against are gitignored and absent in CI.

Verified in both directions, planting a poisoned fixture rather than reasoning about it. With a deliberately corrupt tests/*/.pinakes/index.db in place, all 105 tests across the four affected files pass. Removing the guard from one call site fails exactly that site's test (test_edges.py::test_the_stored_edge_set_agrees_with_the_probe_the_decision_was_taken_on, StoreError), which is what makes the guard's presence the property under test rather than an incidental line.

Generalisable: when a test copies a directory the tool also writes into, the copy must name what it excludes. And a loud environment-coupled failure is not a cheap one — this one cost real time precisely because it arrived wearing the costume of a merge defect, right after a merge.

An exact assertion between two different roundings — green locally, red on one CI leg (20260805 17:55)

MEDIUM — a coin-flip assertion that passed for the wrong reason, and whose failure was earned by an unrelated correct change. test_reports_cores_the_way_macos_percent_converts_to_them read the two numbers off the peak: line and asserted cores == pytest.approx(percent / 100.0) — at approx's default relative tolerance, i.e. effectively exact. But report() renders percent at 0 dp and cores at 1 dp: they are two roundings of one value, not one value printed twice. Exact agreement is a coincidence of the input, never a property of the code.

It held only while a single-process sample sat at exactly 100.0. The tree-sum fix (measured the launcher, not the work) made a one-core loop read 101.4 — parent plus child — and "101"/100 != 1.0 turned CI red.

The tell is which legs failed. check (light pdf) failed; check (light) and check (light pdf claude) passed on the same commit, same code, same test. Three legs disagreeing about one assertion is not a flaky environment — it is an assertion whose truth depends on a measured value nobody controls.

And it failed on merged main, not on the branch. The branch's own ./check.sh was green, twice, because this machine's readings rounded agreeably. A local gate cannot rule out an assertion that is only usually true.

Fixed with abs=0.06 and the arithmetic written down: 0.5 of display error in the percent is 0.005 of a core, plus 0.05 from the cores field's own rounding, so 0.055 is the largest honest disagreement between the two fields.

The loosened bound still bites — verified, not assumed. Dropping the /100 from CpuTrace.cores fails exactly this test (1.0 against 101). A tolerance that admits formatting disagreement while still rejecting arithmetic disagreement is the assertion the test always meant to make.

Generalisable: comparing two rendered numbers is comparing two roundings. Either compare the values before formatting, or state a tolerance derived from the display precisions — never an exact comparison "because they should be equal". This is the repo's recurring class once more: the assertion named "the conversion is right" was actually testing "the two roundings happen to agree".

Mutation testing found a guard that could not fire (20260805 19:07)

LOW as a defect, worth keeping as a method result. The numbered-heading predicate's document-level check was written straight from its own spec, which says the numbers must form a valid outline walk and that no number repeats. Both were implemented: a seen set alongside the step-validity rule.

Mutating the clauses one at a time, eight of nine mutants were killed by the test named for that clause. The ninth — deleting the no-repeats check entirely — broke nothing.

The first instinct is "write the missing test". The right answer was that no such test can exist: every step the walk permits raises the number tuple lexicographically — a sibling raises its last component, a first child appends to it, an ancestor's next sibling raises a shallower one — so an accepted sequence is strictly increasing and a repeat is unreachable. The check was dead code wearing a guard's clothes.

It was removed rather than kept as defence in depth, and the reasoning put in the docstring. A guard that cannot fire still reads as one, and the next person to touch the step rule would weaken it believing this had their back. The spec keeps the no-repeats sentence — as a statement of intent it is correct, and the implementation note now says why it needs no code.

Generalisable: a surviving mutant asks a question before it asks for a test — is this reachable at all? Adding a test for unreachable code is how dead code acquires the appearance of coverage. This is the inverse of the failure this project keeps meeting: usually an assertion is satisfied by something other than the property it names; here a guard was satisfied by something other than itself.

Running it found what reading it could not (20260805 19:15)

MEDIUM, and the reason it is recorded beside the note above: the two findings came from opposite methods on the same increment. The dead guard was found by mutating code. This one was found only by building a KB and using the feature the way a user would.

[chunking] headings = "numbered" added to an already-synced KB, then a plain pnk sync:

	result
plain pnk sync	1 unchanged · every heading_path still empty
pnk sync --rebuild	1 indexed · the three heading paths, and the first parent-child edge

An incremental sync re-chunks a document only when the document changed. A manifest-only edit changes no content hash, so the feature silently does nothing — and pnk doctor then reports exactly the condition the user just tried to fix.

Every test passed throughout, and no test could have caught it. The unit tests call chunk_document directly with the parameter set; the mechanism that drops it lives in sync.py's change detection, one layer up. The defect is not in either component — it is in the seam, and a seam is only visible from outside both.

It is also pre-existing: max_tokens and overlap have always behaved this way. Three releases did not surface it because no [chunking] key had ever been worth flipping on a KB already indexed. Adding the first one that is, is what made an old defect newly reachable — and "my change did not cause this" is not the same as "my change did not make it matter".

Generalisable: for any change that adds a knob, turn the knob on a real KB before landing. Unit tests verify a component honours a parameter; only using it verifies the parameter arrives.

A warning that cleared itself without the fix being applied (20260805 20:20)

HIGH — the first draft turned a silent defect into a lying one, and every test passed. The chunking-drift warning was correct. What was wrong sat 300 lines away: sync.py wrote the current chunking identity into meta at the end of every sync, including the incremental one that had just refused to re-chunk anything.

So the sequence was:

step	what the user saw	what was true
edit manifest, pnk sync	1 unchanged + the new warning	index still built the old way
pnk sync again	1 unchanged, no warning	index still built the old way
pnk doctor	OK chunking coherence	index still built the old way

A warning that clears itself without the fix being applied is worse than no warning: it converts "the tool said nothing" into "the tool said it was fine". The index actively claimed a coherence it did not have.

Found by running it a second time, not by testing it. The unit tests asserted the warning appears — it did. Nothing asserted it persists, because persistence only fails on the second invocation, and a test that runs an operation once cannot see a defect that needs it twice. The fix's own test is now ..._persists_until_the_rebuild_actually_happens, which syncs three times.

The correct rule turned out to be narrow: record the identity only when every chunk in the index was produced by this run — a rebuild, or a first build into an empty index. An incremental sync re-chunks only what changed, so after one the index is a mixture, and there is no single honest value to record. Leaving the old value is right: it keeps warning, which is exactly what a mixed index deserves.

Generalisable, and it is the second time today: a state-writing side effect belongs with the work it describes, not with the command that happened to run. set_meta is called once per sync and was treated as "the place identity goes" — but identity is a claim about the chunks, and only one of those code paths actually produced them all.

The corpus rejected two of my fixes, and that is the result (20260805 21:00)

The measurement §5.3 demanded was run, on real RFCs, in doubling rounds: 66 → 131 → 259 → … The rule was state the predicate first, measure second, and treat a poor match as a finding rather than a licence to loosen a clause. It held, twice, in the direction that costs something.

Round 1 (66 documents) found a false positive. RFC 769 lists facsimile command codes as 56 - SET-UP, 57 - DATA, 58 - END. Consecutive integers, short labels, column 0, blank lines around — every clause passed, and the predicate produced three headings that are not headings.

The first fix was wrong, and the corpus said so. "A heading's title must not begin with punctuation" kills it. It also killed three genuine documents: 5.1. /get, 2.7.3. "iprev", and RFC 2010's entire outline, which numbers real sections 1 - Rationale and Scope — the identical shape as the false positive. Form cannot separate them. What separates them is where they start: an outline begins at section 1, a list of opcodes begins at 56. That rule changes exactly one verdict across the corpus, and it is the wrong one. It shipped as clause 9.

Round 2 (131 documents) found a second false positive, and rejected a second fix. RFC 778 numbers a procedure — 1. Connect to COMSAT-GAT host…, 2. Send the command… — starting at 1, so clause 9 does not catch it. The obvious discriminator is that a heading stands alone: require a blank line after the candidate. Measured, it removed the false positive and four genuine documents with it, because real headings wrap:

7.4.  The Network Information Center and
      Requests for Comments Distribution Contact

Rejected, and RFC 778 is recorded as an accepted bound instead. Labelling the steps of a numbered procedure as sections is defensible; 56 - SET-UP was not. Not every false positive is worth a rule, and a rule that costs real structure to buy a marginal one is a bad trade even when its net count looks fine.

What the corpus did buy: clause 10. A recurring convention numbers top-level sections 1.0, 2.0, mixing the two freely — RFC 2006 runs 6 then 7.0, RFC 2024 runs 1.1 then 2.0. Read literally those are depth changes no walk can accept, and the document is rejected whole. Normalising a trailing zero fixes it, and is safe precisely because a real subsection never carries .0.

Clause 10's own test then caught a bug the walk could not see. The walk normalised 2.0 to 2 and accepted the document — while the heading stack still used the raw depth, nesting 2.0 under 1.0. The document passed and the hierarchy was wrong. Two places consume a number's depth and only one had been taught the convention.

Generalisable, and the reason the doubling protocol matters: clause 9 was derived from 66 documents and looked complete; 131 documents produced a false positive it could not catch. A fix validated at one corpus size has been validated at one corpus size. Every round both re-checks the previous fixes and gets a vote on the next.

The sync-CPU number, and the instrument proving itself in the field (20260805 21:56)

The measurement item 3 demanded since 20260804 was finally run, and it reverses the item's own framing. 55 modern-era RFCs — 16 557 chunks — rebuilt under fastembed:

wall-clock	1 497.7 s (~25 min), 1 451 samples
peak	500% — 5.0 of 10 cores
mean	480% — 4.8 of 10 cores

The loop is serial and the backend under it is not. sync.py embeds one document at a time, so the loop is single-threaded — but ONNX Runtime is already using half the machine beneath it. Item 3's own fork therefore resolves against the change it was written to consider:

The backend already saturates the machine → the loop is fine, and the win is a bigger batch (embedding several documents' chunks in one embed() call), not processes.

It also lands exactly on the trap that item named: "do not stack a process pool on top of a threaded backend". At 4.8 cores already consumed, two workers would take ~9.6 of 10 and anything beyond oversubscribes. The intuitive fix — a pool sized os.cpu_count() - 1 — would have been nine workers on a machine with room for two.

The instrument proved itself in the field, and it is the reason to trust the number. Sampled live from the same process tree:

process	%cpu
measure_sync_cpu.py	0.7
uv run	0.0
the actual pnk sync python	491.9

The pre-fix tool watched the launched pid — uv run — and would have reported 0.0 cores for a workload using five. That is not a number anyone would have questioned: it is the finding item 3 went looking for, and it would have licensed exactly the process pool this measurement rules out. A tool whose failure mode is "confirms your hypothesis" is the most expensive kind.

Bounded, and the bound is stated rather than buried: fastembed only. sentence-transformers needs the 2 GB [st] extra and is unmeasured, so nothing here licenses a claim about torch.

A second number fell out of the same run, unasked: 15 559 of 16 557 chunks carried a heading_path — 94%. The corpus that opened this whole line of work indexed 106 806 chunks with zero. The grammar works on real documents, and that is the first evidence of it outside a test.

Removing a blanket guard exposed what it had been incidentally protecting (20260805 22:11)

MEDIUM — and the finding came from an existing test, not from reading the diff. pnk init's emptiness check was removed so a directory with content could be adopted. test_ci.py then failed: it asserted that an existing .github/workflows/pinakes.yml is refused, and its evidence was the string "not empty".

The obvious read is "update the assertion". The real finding is that the emptiness check had been holding a second job nobody had written down. write_workflow does refuse to overwrite a hand-edited workflow — but it runs after pinakes.toml has been written. With the emptiness check gone, that refusal would leave a half-made KB: a manifest the user never asked for, and a re-run that now fails with "already a KB", whose only way forward is deleting a file init created itself.

Moved to _check_target, before any write, with the failure mode in the comment. The test now asserts "already exists" — more precise than "not empty", and it names the only thing actually in the way.

A second decision was refined in the same pass, and it is recorded rather than quietly taken. The decision as written said "add a refusal naming any file init would write that already exists". Implemented literally, that refuses on README.md and .gitignore — which a real repository always has — so adoption would still have been impossible in exactly the case the item exists for. The intent was "do not destroy the user's files"; the implementation honours it by never overwriting and reporting, which is strictly safer than refusing and actually achieves the goal. --ci is the one exception, because it is an explicit request rather than a side effect.

Generalisable: a coarse guard removed is not one behaviour removed. Before deleting one, ask what else has been quietly standing behind it — and take the failing test that follows as evidence about the system, not as a chore.

Slimming CLAUDE.md — a relocation's real cost is its pointers (20260806 00:33)

HIGH — the reference sweep has to run on the source file's name, not on the text that moved. Extracting two sections out of CLAUDE.md left seventeen citations across the tree pointing at content the file no longer carries. Not one of them quotes the moved wording; they name the file — `CLAUDE.md` calls "the free path stays free" non-negotiable, "the invariant CLAUDE.md gains says so explicitly", "for the reason CLAUDE.md states". A grep for the moved sentences finds zero of them, which is exactly what "keep docs in sync — grep for what changed" instructs you to run.

The two sweeps measured the difference:

Sweep	Scope	Found
Neighbourhood audit, on the moved terms	docs/, tools/	4
Second pass, on the string CLAUDE.md	whole tree	13 more, all in src/ and tests/

src/ and tests/ were never opened by the first pass, because a docs-only change does not look like it touches them. They held the majority: embed.py, sidecar.py, cli.py, ids.py, extract/claude.py, extract/pageyield.py, budget/ledger.py, and five test docstrings, each citing an invariant that had just moved to docs/INVARIANTS.md. The rule now: after relocating anything out of a file, grep -rn '<that filename>' over the whole tree and re-judge every hit — a pointer names its target, so it is invisible to a content-based search.

MEDIUM — docs/INVARIANTS.md is an index because the facts already had owners. Before extracting, each of the nine invariants was checked against the docs that would hold it: eight were already stated in DESIGN.md (§1 allowlist, §2.2 sidecar, §3 storage/schema, §5 ledger and Decimal), MANIFEST.md (the id rows, the round-trip bounds table), VERIFICATION.md (§ The sidecar round-trip) or CLI.md (--clear-cache=paid). A verbatim move would have created a second copy of eight facts inside the file set whose stated rule is one fact, one home — and a second copy drifts silently, because nothing compares them. So the page links each owner and writes out only the five implementation rules nothing else states.

MEDIUM — the no-loss check has to be mechanical. Normalising every sentence of the old file and matching it against the new homes flagged 29 candidates: 25 were false positives and 4 were real losses — a measured number (980 RFCs), the "first time since it opened" qualifier, a plan's self-description, and the why behind read the clock, never compose a timestamp, whose only in-repo home had been the sentence being deleted. Re-reading the diff would not have separated those four from the twenty-five; only a per-sentence check did.

Not fixed, out of this change's scope: tests/test_extract_claude.py and tools/record_claude_fixtures.py both still require a local YYYYMMDD HH:MM for a recording's --at, against the repo's UTC rule adopted 20260804 11:32. One for plans/20260731_1202-open-corrections.md.

A plan's blocking premise was inferred, not measured — and it was wrong for two days (20260806 04:05)

HIGH — the reason a plan gives for a blocker is reused long after the blocker is gone. plans/20260805_1721-metadata-as-retrieval-context.md said tests/demo-kb could not measure the injection experiment because "no section spans multiple chunks, so there are no continuation chunks to rescue — the mechanism has nothing to act on". Measured with the real chunker and the real tokenizer over all 30 documents: 30 of 30 sections span more than one chunk, and 29 of the 30 continuation chunks do not contain their own heading text. The corpus carries the exact mechanism the plan said it lacked.

The error was an inference from the wrong mechanism. The plan's fact was right — the documents are ~7 lines — but it reasoned that a short document fits in one chunk. chunk.py splits on paragraph blocks first (Block is "one paragraph under one heading path") and only then applies token limits, so a 7-line document with two paragraphs yields two chunks of 27 and 31 tokens under a 120-token budget. Nothing about the token budget was ever reached.

What makes this worth recording is that the plan's conclusion was right. Use the RFC corpus — correct. So the wrong premise cost nothing at the time and would have cost a great deal later: it is the sentence a future agent quotes when deciding whether some other experiment can run on the demo KB. A conclusion is checked when it is acted on; a reason is copied forward unexamined.

MEDIUM — the honest reason demo-kb cannot license this is arithmetic nobody had done. 66 answerable questions, 4 misses, and 56 of 62 hits already at rank 1. The whole improvable pool on recall@k is 4, and the project's own sign_test(4, 0) returns p = 0.0625 — so a perfect result fails the p < 0.05 bar the graph channel was held to. That is a power limit, not a mechanism limit, and the two have different remedies: a mechanism limit is fixed by a different corpus, a power limit by a different metric or more questions. Writing the wrong one down sends the next person to the wrong fix.

MEDIUM — reasoning from a committed artifact nearly produced a confident wrong number. The miss count above was first read out of tests/demo-kb/eval/outcomes.json, whose header predates G5 — no graph_channel, no edge_kinds, no retrieval.adjacent_k, and graph_gate.read_leg reports its channel as (absent). Two independent reasons the number could not be trusted from that file alone: the artifact is only rewritten under --write-baseline, so CI never refreshes it; and compare() tolerates a ±0.02 drift, which spans 4 to 6 misses — and sign_test(5, 0) = 0.0312 passes. The claim "a perfect result cannot license" was therefore one question wide. Re-running the eval settled it: aggregates identical, and all 74 rows match the committed file exactly. The rule is not "distrust artifacts" but an artifact whose header does not match the current binary is evidence about the past — the identity check graph_gate.check_identity performs on legs is the same check a reader owes any committed number.

MEDIUM — four silent failures sat in front of an experiment costed at "~2 h rebuild + eval". The RFC corpus stamps no max_tokens, so the default 510 applies against a measured window of 512 with 2 special tokens: zero headroom, and prepending anything pushes every full chunk past the window. Embedding an over-length string raises no warning and no error (measured, empty warnings list), and assert_chunkable cannot catch it because it validates max_tokens, never max_tokens + prefix. The direction is what makes it dangerous: truncation removes text from exactly the long chunks the hypothesis is about, biasing the result toward no movement — a false negative that reads as a clean result. Separately, the lexical channel cannot be injected at all without a new chunks column, rewritten FTS5 triggers and a schema_version bump, because chunks_fts is an external-content table filled by triggers copying new.text.

The estimate was not wrong about the run; it costed the run alone. Step 2 is three increments and a measurement, and none of the three was visible from the plan as written.

LOW — a plan that forbids retrying below "the threshold" has to name it. The document's anti-circularity clause read "a result short of the threshold is reported rather than retried" while nothing in it ever said what the threshold was. An unfalsifiable gate is not a gate; it was fixed by decision rather than discovered, but it survived a full adversarial review of the plan because the sentence sounds like a commitment.

2a — Real titles and a measured chunking reserve for the RFC corpus (20260806 06:12)

The plan's worked reserve was wrong by more than twofold, and only measuring found it. The injection experiment's plan offered 480 as the corpus max_tokens, from "a measured max prefix of 30" — RFC 9110's largest title > heading_path with section numbers stripped. Measured properly, over every heading path of RFCs 8600-8799 (195 documents, 5 of the 200 numbers unpublished), tokenised with the corpus's own BAAI/bge-small-en-v1.5: the largest prefix is 68 tokens, the per-document largest has median 31, p95 51, p99 61. The median document already exceeds 30. RFC 9110 was an unrepresentative sample for one reason nobody spotted: its title is two tokens long, and titles run to 32.

Reserving 30 would have truncated roughly half the corpus's longest chunks — silently, since an over-length embedding input raises no warning and no error. That truncation removes text from exactly the continuation chunks the hypothesis is about, so it biases toward no movement: a false negative that reads as a clean result. The reserve shipped is 96, deliberately 41% above the measured maximum, because 200 numbers is under a third of the modern band.

The plan said "do not re-derive these" about a table whose own line numbers had already drifted. Same failure class, one level up: a number recorded as measured invites reuse, and reuse is exactly what makes a sampling flaw permanent. The four "read this before writing a line" findings in §2 were all correct and all load-bearing; the one worked number beside them was not.

A defect I nearly introduced, caught in my own adversarial pass. Preserving the KB id across a re-run — needed once the output directory holds permanent document ULIDs — makes a re-run look identity-preserving while write_kb still rewrote pinakes.toml. That would silently discard the [retrieval.confidence] thresholds 2c will fit onto this corpus, with every command reporting success: the same KB by its id, no longer the same KB by its calibration. The fix removed the reason rather than the symptom — an existing manifest is not rewritten at all, so the id is stable by construction and the existing_kb_id reader that motivated the concern was deleted. Its cost is that a re-run does not adopt a changed [chunking], so the run now names both values.

What nothing else in the repository exercises: a title with YAML punctuation. RFC 8713 is "IAB, IESG, IETF Trust, and IETF LLC Selection, Confirmation, and Recall Process: Operation of…". Every committed corpus is hand-titled in plain words, so the first colon in a title: arrives from the RFC Editor — into the one file holding a document's permanent ULID. ruamel quotes it and it round-trips; that is now asserted through sidecar.read rather than assumed.

Verified by execution, which the plan had none of. Two RFCs built and synced with the real fastembed backend: both documents indexed under their published titles, 801 chunks, largest token_count exactly 414 — so the stamped cap binds on real text, as finding 1 said the default 510 does against a 512-token window.

A bound left in place, not fixed. corpus.json describes the run, not the directory: a re-run over a smaller --rfcs set records fewer RFCs than docs/ still holds. Pre-existing, and now slightly more visible because titles.kept_from_earlier_run hints at accumulation. Worth fixing before the run at 2f, where the corpus's composition has to be attributable.

2b — Refusing a prefix that would not fit (20260806 08:24)

A refusal with no caller is a refusal nobody has watched fire, so this one was written to be runnable on day one. The manifest option that turns injection on belongs to 2d, so nothing on the indexing path calls assert_prefix_fits yet. The temptation was to ship the check alone and let 2d supply the thing that exercises it. What shipped instead is the whole prefix construction — metadata_prefix, embedding_text, and the unnumbered_heading_path they read — so the refusal measures exactly the string that will later be embedded, and every part of it is reachable from a test today. 2d then adds one option and one call, not a mechanism.

The additive estimate was measured rather than argued. The check counts each distinct prefix once, with its separator attached, and adds chunk.token_count rather than re-tokenising every injected string — a document has orders of magnitude fewer heading paths than chunks. That is only safe if the sum never under-counts the concatenation. The reasoning (a tokenizer that splits on whitespace before merging cannot produce more tokens from prefix + sep + text than from its parts) is sound but is the kind of reasoning that is wrong once per project, so it was run: against BAAI/bge-small-en-v1.5, over 43 503 chunk/prefix pairs from 195 RFCs, the estimate was exactly equal to the concatenation's real token count every time — not merely bounding. The same run reproduced 2a's corpus figures from an independent code path (largest prefix 68, per-document largest median 31 / p95 51 / p99 61, longest title 32) and confirmed the refusal fires for 195 of 195 documents at the default max_tokens = 510 and for none at the corpus's 414.

The reserve is checked, not the worst chunk in hand, and that was a decision. Per-chunk pairing is more permissive and more exact: it refuses only what would actually truncate today. It was rejected because the two legs of an A/B comparison must chunk under the same max_tokens or they are different corpora — so what has to be safe is the setting, not this morning's text. A document that passes because none of its chunks happens to reach the cap would start truncating on the next edit, mid-experiment, silently. Refusing the setting is stable across documents and across edits to them.

A field added to a frozen dataclass is a field somewhere else forgets to copy. _with_pages rebuilt every PDF chunk field by field to attach page numbers, so unnumbered_heading_path would have arrived as None for PDFs only — green suite, silent gap. It now uses dataclasses.replace, which cannot omit a field, and a test pins the property. The same reasoning kept the field out of as_row: the stored form is the citation form, and a second column is a second thing to keep in step with it.

Markdown keeps whatever number its author typed, and that is the same rule rather than an exception to it. ## 1. Introduction yields 1. Introduction in both paths, because nothing parsed a number there — # is syntax and is already gone, and the text after it is the author's. Only the grammar that parsed a number is entitled to remove one. A regex over the joined string would have been shorter, would have drifted from the grammar's own rule, and would have eaten the 404 from # 404 Not Found.

Two findings from the increment's own adversarial pass, both about a value with more than one home. The first: the prefix's join was introduced as a new constant, which would have made four literal " > "s in this module plus graph/edges.HEADING_SEPARATOR reading them back — for a format that is persisted in chunks.heading_path, where a disagreement empties three edge kinds and reports nothing. It is now one HEADING_JOIN used at every site that builds a heading path, with the consuming copy named in its docstring. The second: title is the user's field in a hand-edited sidecar, so it can be "" or " ", and the first draft treated whitespace as content — injecting a separator with nothing in front of it into every chunk of that document. Neither would have failed a test or a run; both are the shape this project keeps meeting, a value that is wrong in a way nothing reports.

2c — A golden set authored by agents that did not know what it was for (20260806 11:42)

The anti-circularity rule was implemented rather than promised. The plan requires the questions frozen before any injection code exists, "so that no number can influence them" — and the deeper risk it names is fitting the question set to the mechanism, which "is undetectable afterwards". An author who knows the hypothesis cannot prove they ignored it, and no reviewer can check it from outside. So the set was authored by six agents over disjoint document slices, each told only that it was writing an evaluation set for a retrieval system and forbidden from reading this repository, where the plan would have told it. Blind authorship is the only version of that guarantee that survives review.

The exit criterion failed the first time, and how it failed was the useful part. 70 questions produced an improvable pool of 9 against a criterion of 10. The shape mattered more than the number: 51 of 60 answerable questions were already at rank 1, lexical and simple-lookup both scored 1.00, and 8 of the 9 pool members were paraphrase. On a corpus of distinctive technical vocabulary — protocol acronyms, registered code points, field names — BM25 with a reranker essentially solves the classes that share words with their document.

The obvious fix was the wrong one, and it would have passed. Authoring more paraphrase questions reaches a pool of 10 quickly. It also enriches the set with exactly the class a change to the embedded text is most likely to move, which is fitting the instrument to the hypothesis one step removed: the criterion would pass and the result would mean nothing, with nothing in the artifact to show it. The two new slices carried the same proportional mix as the first four, over 135 documents no question had touched. The pool went to 15, and it changed shape — 11 paraphrase, 2 lexical, 2 simple-lookup, where before it was one class. A pool spread across classes is what makes compare()'s per-class guard able to catch a change buying one class out of another.

The expensive error in a question set is self-concealing, so the set carries its own evidence. A question pointing at the wrong document looks exactly like a retrieval miss. It would have inflated the pool and made the power criterion pass for the wrong reason — the measurement then resting on questions nobody could answer either. Every answerable question therefore records the sentence from its document that answers it, verbatim, and tools/verify_rfc_golden_set.py refuses the set if a sentence is not there. All 96 verified. Whitespace is normalised because RFC bodies are hard-wrapped at ~72 columns, which also means the recorded evidence is often the fragment on one line rather than a whole sentence.

A slice can only check its own slice. Each author confirmed its no-answer topics absent from its own 50 documents. Checked across all 195, three had mentions elsewhere — "camel case" in rfc8618 against a question about the CAMEL protocol, one mention of blockchain databases in rfc8673, Bluetooth as a transport example in rfc8628 and rfc8793 — and later, SCADA as a category label and NFC as both a URI transport and Unicode Normalization Form C. None answers its question, and all were kept deliberately: a calibration set wants plausible near-misses, and a lexical collision that answers nothing is exactly that. Two topics were duplicated across slices and dropped.

filter and multi-hop are absent by decision. filter needs metadata worth filtering on and these sidecars carry a title and nothing else. multi-hop is the graph channel's class and graph_channel is off here, so such a question would score as an ordinary lookup while being reported under a name claiming otherwise.

The corpus is not committed, so everything that reads it had to be. The questions, the fitted confidence thresholds and the before leg all live in the repository; the 195 documents do not. That is the same lesson build_rfc_corpus.py was written for — a 300-RFC corpus once produced this project's most useful finding and died with the machine that held it. The thresholds are stamped into the builder's manifest template rather than pasted into a generated pinakes.toml, which also makes both legs of the comparison fitted identically by construction: refitting after a change would measure the refit.

A defect found by asking what would protect the artifact this increment produces. Capturing a before leg raises the question of what stops it being compared against an after produced differently — and graph_gate.check_identity turned out to guard only the graph channel's own settings. Reading eval.header for what it does record showed the gap was one level lower: the function's docstring promises "every setting that can move a row", and [chunking] was not in it. Every other outcome-deciding setting was. So the leg committed here could not have been shown to have been chunked at 414 rather than the default 510 — the precise confusion 2a's reserve and 2b's refusal exist to prevent, surviving into the artifact that records their result. Three fields, no schema bump, because no row changed.

2d — The screen said no, and the controls are what make that worth believing (20260807 09:07)

The result: 6 improved, 6 regressed, 84 unchanged, over 96 answerable questions. The pre-registered criterion was strictly more improvements than regressions, so the screen is a no-go and the schema_version bump at 2e is not taken. Per the pre-registration these numbers are the whole report — they are not evidence for or against the hypothesis in either direction, and they appear here and in the increment's commit message and nowhere else. The screen's own artifacts were deliberately not committed.

What was measured, stated exactly, because a null is only as good as its controls. Injecting title > heading path into the text that is embedded — the vector channel alone — moved nothing net on this corpus at rerank = "none". The both-channel form was not tested: the lexical channel needs a new chunks column and a schema bump, which is the cost this screen existed to decide. The dilution objection that disqualified vector-only as a gate applies in full to this null: RRF fuses an injected vector channel against an unchanged BM25, so a real effect is attenuated before it reaches a rank.

A null result is a claim about the world only if the instrument was pointed at it, so four controls were run before the comparison — three of which the plan did not ask for.

Control	Result
The uninjected index still reproduces 2c's baseline (rerank = "local")	110 of 110 rows identical, twice — once on main's binary, once on this branch with the option off
Both legs are the same corpus	195 documents, 43 353 chunks, and one sha256 over every chunk text, equal
The injection actually reached the vectors	mean cosine 0.8398 between the before and after vectors of 2 000 sampled chunks; zero unchanged
The prefix was the intended string	195 of 195 published titles, zero filename stems — finding 5's confound absent — and 93.2% of chunks carrying a heading path

The third is the one that decides how to read the null. Chunk texts are byte-identical between the legs by construction, so if the injection had silently not happened, every artifact in the experiment would look exactly as it does now — same corpus, same questions, a clean flat result — and the conclusion would be drawn from a no-op. Measuring the vectors is the only thing separating "no effect" from "no injection", and it cost one script.

The option-off path being a verified no-op is not a formality either. The same binary that carries the injection reproduced the frozen baseline row for row, which is what licenses comparing this branch's after leg against a before leg captured on it.

The shape of the movement is more interesting than the count, and it is not being reported as a result. Of the 6 improvements, 5 were paraphrase — the class the hypothesis targets, and the only class with power on this corpus. Of the 6 regressions, 2 were simple-lookup questions that had been at rank 1. That is what a dilution cost looks like: a prefix adds tokens to a vector whose question needed none of them. Twelve of 96 rows moved, so the mechanism is doing something; it is simply not doing more good than harm through one channel. This is an observation about a measurement that was pre-registered not to be interpreted, and it must not become the premise of a retry — the anti-circularity rule says a result short of the threshold is reported rather than retried with a different injection format.

Two things surfaced by building it that the plan had not anticipated.

• The refusal is a per-document failure, not an aborted run. assert_prefix_fits raises a ChunkingError from inside _index_document, and every PinakesError there is already caught by sync's per-document handler: the transaction rolls back, the document is named in the report and recorded in the index for pnk doctor. That is the right shape — one pathological heading path should not cost a 195-document corpus its other 194 — and it still removes the silent truncation, because a refused document is not indexed at all. But "refuses the corpus" was the plan's phrasing, and the test written for it originally asserted a raise that never comes.
• With injection on, every document is prefixed, because skeleton() falls back to the filename stem. A document can reach the embedder with no heading_path; it cannot reach it with no title. So on an uncurated corpus the injected string is a filename — finding 5's condition, now located at the sync boundary rather than in the abstract. It is why the RFC corpus mints published titles before its first sync, and it is the strongest argument for the option defaulting off.

A test that passed vacuously, caught during development and worth naming. The assertion that the injected prefix uses the sidecar's title was written over a sidecar-only edit — which is a RefreshMetadata action, so it updates the row and re-embeds nothing. all(...) and not any(...) over an empty list are both true, and the test was green while proving nothing. The fix was --rebuild plus an explicit assert backend.embedded precondition. Any assertion of the form "everything embedded looks like X" needs a companion assertion that something was embedded.

tools/two_leg_gate.py exists because the instrument had the same gap one level up. 5993521 made an eval artifact record the chunking it was produced under; nothing compared it — graph_gate.check_identity takes three legs shaped to the graph channel and inspects k, embedding, rerank, ranking and retrieval, but not chunking. Two legs chunked at different max_tokens therefore compared clean, which on one RFC is 63 of 1 858 chunk texts differing: a rechunk reported as the effect under test. The new tool refuses on any header difference outside one named key, and it excepts that key by path, not by block — excepting the whole chunking table would hide exactly the rechunk it is there to catch.

2d review — the grep that was necessary and not sufficient (20260807 09:54)

HIGH — a rebuild could leave a half-injected index, and the check that should have caught it was looking for the wrong thing. Before landing the injection, this increment verified that sync.py has exactly one .embed( call on the indexing path — grep -rn '\.embed(' src/ returns that one, plus two query-side calls. The conclusion drawn was "every vector on the indexing path goes through the switch". It does not follow, and the counter-example was already in the file: _copy_forward_protected_document writes rows into embeddings with an INSERT … SELECT from the index being replaced. It produces vectors without embedding anything, so no grep for .embed( could ever have found it.

The consequence was the failure class this project exists to prevent. A KB with a paid-extracted PDF, [chunking] metadata flipped to "prefix", and pnk sync --rebuild: the protected document keeps its uninjected vectors, set_meta stamps chunking_metadata = "prefix" over the whole index, and the next pnk sync and pnk doctor both report no drift. Every command succeeds, and half the index is injected.

The fix separates the two costs that had been treated as one. The docstring said the document is "never re-extracted, never re-embedded", as though those were the same protection. Extraction is what spends money; embedding is local and free, and the chunk texts are carried forward anyway. So the chunks are still copied and the extraction still never re-run, and the vectors are recomputed under this run's settings — which fixes both directions, since turning injection off again had the mirror-image defect.

One guard is deliberately louder than it needs to be today. unnumbered_heading_path is not persisted (chunk.Chunk says why: the stored form is the citation form, and a second column is a second thing to keep in step). A carried-forward chunk therefore cannot say what its path looks like with the section numbers removed, so injecting the stored form would quietly prepend the citation form this experiment measured at 44% numbers and rejected. Unreachable now — only PDFs are ever protected and the PDF path records no heading path — but step 5 of this plan is PDF layout heuristics, which is exactly what would make it reachable. It refuses with a named remedy instead.

What the review round found beyond that, all of it in code this increment wrote:

--sign-test printed FAIL at 0.05 and exited 0	the flag names the 2f gate, which licenses the irreversible schema bump; a driver branching on $? would have taken it
A miss was written to the artifact as Infinity	invalid JSON: JSON.parse rejects it, jq silently coerces it to 1.8e308 — the one outcome the rank ordering exists to make visible became a very good rank
A truncated artifact exited 1, same as a genuine no-go	read_outcomes only refuses a file that parses; a JSONDecodeError is a ValueError, which the first version of the handler did not catch
Nothing recorded which leg was which	transposing --before/--after inverts the verdict, and the identity check cannot catch it — it is never told which value is the baseline, only that the two must differ
pnk doctor's half of the drift promise was untested	a mutant reading a constant there passed the entire suite
Two more vacuous assertions	backend.embedded == [rows] is [] == [] when a run indexes nothing; all(…) over an empty list again

Three of those six are the same defect in three costumes, and it is worth naming once. An assertion of the form "everything we produced looks like X" is silently satisfied by producing nothing. It has now been found three times in this increment alone — once during development, twice in review — so the rule is: any assertion over a collection the code under test produced needs a companion assertion that the collection is non-empty, and preferably one that names the expected count.

The review that found this was itself a measurement, and it half failed. Five adversarial lenses were run as independent agents; a usage limit killed 15 of the 17 agents mid-flight, including every verifier. The workflow returned {"confirmed": [], "refuted": []} — which reads exactly like a clean review and was nothing of the sort. The findings were recovered from the agents' own transcripts and verified by hand. An empty result from a harness that partially failed is not evidence of absence, and a report that does not distinguish the two is worse than no report: confirmed: [] was one careless sentence away from becoming "the review found nothing".

2d review, round two — the argument for the key's placement was false when written (20260807 10:34)

HIGH — the option was put in [chunking] rather than [retrieval] because [chunking] is recorded in the index and therefore cannot flip silently. That reasoning was correct about the mechanism and wrong about every KB in existence. chunking_drift treats a key absent from meta as unknown, never drifted — the rule that stops an upgrade demanding a rebuild of every index — and chunking_metadata is absent from every index built before this release. Only a --rebuild ever stamps the chunking identity, so the absence is self-perpetuating. Measured on a KB built by this branch with the key then deleted, which is exactly the shape a 0.15.1 index has:

drift ()            embedded 0          backend calls 0
pnk doctor:  OK  chunking coherence: index matches the configured chunking

An affirmative OK over uninjected vectors, forever. For a pre-existing index the two manifest sections behaved identically, which is the thing the placement argument was written to rule out.

The fix turns on a distinction the original rule did not need. max_tokens and overlap have been settable since v0.1, so an index that fails to record them could genuinely have been built under any value: absence there is ignorance. chunking_metadata is different — no release that could have written any existing index was able to inject anything — so absence proves off. store.ABSENT_MEANS says so, and because it resolves to the default, it fires only for a user who explicitly opted in: the compatibility guarantee it looks like it threatens is untouched. Both directions have tests, and the second one matters as much as the first — an unclearable warning on every upgraded KB is the failure mode the heading-coverage check already had to answer for.

MEDIUM — the fix for round one's finding shipped with the same class of hole inside it. Round one found that --rebuild carried a protected document's vectors forward, and the fix re-embedded them. But the re-embed called embedding_text without calling assert_prefix_fits, so the one path that re-embeds without re-chunking became the only path with no truncation guard — and it is the path that needs one most, since its chunks were sized by whatever max_tokens built the previous index and are never re-chunked. A fix for a silent-truncation defect that reintroduced silent truncation one function away.

MEDIUM — and it also introduced a way to publish a half-written document. DETACH requires the transaction closed, so that function commits in a finally; with the writes inside that block, a document whose embedding failed was committed active with chunks and zero vectors, which _apply's rollback() could no longer undo and --rebuild's unconditional index swap then published. No later sync repairs it: the file's content hash is unchanged, so every future run says Skip. Now the old rows are read under the attach and every write happens after it, in one transaction the caller can still roll back.

MEDIUM — the eval artifact labels a leg from the manifest, never from the index. Every [chunking] value in a header is read from pinakes.toml at eval time. Because flipping metadata changes no chunk's text, hash or span, an eval run against an index that was never rebuilt produces a byte-for-byte plausible artifact stamped metadata: "prefix" over uninjected vectors — and tools/two_leg_gate.py would accept it as the injected leg, since it compares headers to headers. The instrument that licenses an irreversible schema bump could be handed a leg that never existed. eval.run now compares the index's recorded chunking against the manifest and refuses before scoring a single question.

The four rules this round leaves behind.

1. A grep for the operation is not a proof about the outcome. Round one's defect survived a check for every .embed( call site because the offending path produces vectors with INSERT … SELECT. Round two's survived a reading of chunking_drift because the defect is in what the function does with a key that is not there. Both times the search was over the wrong set — the question is never "where is this called" but "what else can reach this state".
2. A fix is a change, and gets the same review as one. Two of this round's findings are in code written to fix round one, hours earlier and with the defect fresh in mind.
3. State an argument in terms of the population it covers. "[chunking] is recorded, so the flip is reported" was true of indexes this release builds and false of every index that existed. The sentence never said which it meant.
4. A partially-failed harness must not report like a completed one. Round one lost 15 of 17 agents to a usage limit and returned {"confirmed": [], "refuted": []} — indistinguishable from a clean review. The findings were recovered from the agents' transcripts by hand; one was the HIGH above. Round two ran complete: 14 findings, 14 judged, 9 confirmed, 5 refuted.

T1 — The version archive, and what mutation testing found in it (20260807 19:36)

HIGH — the plan's own baseline undercounted the sites it had just corrected, and the second count was as wrong as the first. The template-release plan was re-verified against main at 71911e2 on 20260807, and that re-run specifically corrected the notes@1.0 site list: it had said "both, in test_init.py", and the correction raised it to "six sites in five files", with a box explaining that the original defect was a grep scoped to one file. Running the corrected command at that same commit returns nine sites in eight files. The three it still missed were tests/test_sync_links.py:66 and the two committed KB manifests, tests/demo-kb/pinakes.toml and tests/partner-kb/pinakes.toml. The lesson is not "grep wider" — the plan already said that. It is that a count in a document is a measurement, and re-running the command is the only thing that distinguishes a corrected count from a confidently wrong one. The correction was written and believed; nobody re-ran it.

HIGH — a formatter can edit a file the project has promised is frozen. The archive's whole value is byte-identity: _versions.toml records a SHA-256, and pnk upgrade will diff against those exact bytes. check.sh runs ruff format --check . over the whole repository, and ruff reformats Python inside Markdown fences. A template README.md that ever gains a python fence would be rewritten in its archived copies too — the project's own formatter editing a version that already shipped — and it would surface as a ledger mismatch one leg away from its cause. Latent today (no template README has a fence) and closed with a [tool.ruff] extend-exclude. Generalisable: whenever a repository declares some bytes immutable, list every tool with write access to them. The gates were audited; the formatter was not, because it is not a gate.

MEDIUM — mutation testing found a branch no input could distinguish, which is the same defect class as a gate that cannot fire. git_history_reason probed --is-inside-work-tree and then --is-shallow-repository. Neutering the first changed no test result: outside a checkout the second exits 128 anyway, so the first was dead weight wearing the clothes of defensive care. It was removed. This increment exists because pnk doctor carried a check that could never fire for eleven releases; shipping a redundant branch inside its fix would have been the same mistake at one-tenth the scale. A branch that no test can tell apart from its absence is a branch nobody knows works.

MEDIUM — the first mutation run was invalid, and its invalidity was informative. Three mutants changed content_hash and turned the whole suite red rather than one test, because the committed ledger pins the hash function: change it and leg (iii) fires everywhere. That coupling is a feature — the hash definition cannot drift silently — but it destroys the per-assertion signal, so the harness was corrected to regenerate the ledger under each hash mutant, which is the tree such an implementation would really have shipped with. Separately, two tests both tampered with README.md, so a mutant exempting the README failed all three. A mutant that travels to another test means the two tests share a vector, not that either is wrong — but the sharing is worth removing, because it makes every future mutation result harder to read.

MEDIUM — leg ordering decided which remedy the user is given, and the wrong order gives the opposite one. Editing a frozen _versions/<live>/ file also makes the live files differ from it. With the live-vs-archive comparison first, the gate reported "the live files differ from archived 1.1" and advised bumping the version — when what happened was that a published version was edited and needs restoring. The ledger check now runs first, so each fault reports its own leg. Found by running the failure paths, not by reading the code: both orderings look correct on the page and only one is.

LOW — "exactly one commit" was unsatisfiable in the increment that introduces it. Leg (vii) as specified fails on its own landing commit, because ./check.sh runs before the archive is committed and git log returns zero. It is at most one: zero means not yet committed, one means added and never touched, two or more is the violation.

HIGH — the gate reproduced, inside itself, the exact defect it was built to catch. Given --templates as a relative path, leg (vii) built its git log pathspec against the process working directory while git resolved it against the templates directory. It matched nothing, git log returned empty, zero commits read as "not committed yet", and the gate printed history leg (vii) ran … none edited and all legs green over a tree carrying the coordinated three-file edit. That is the strong mode claimed while nothing was checked — the thing this increment exists to stop pnk doctor doing. Every leg-(vii) test passed --repo explicitly, so nothing covered it. A flag that changes where a tool looks needs a test that runs it from somewhere else; resolving paths once at the argument boundary is the fix, and the general rule is that a relative path and a cwd argument must never be chosen independently.

HIGH — leg (vii)'s first design blocked the project's own procedure, and no single mechanism fixes it. Counting every commit that touched an archived directory fails a branch that adds one and then corrects it during review — which is exactly what docs/BUILDING.md requires (green ./check.sh before review, review fixes in their own commit). The failure text said the archive "still says what the version said when it shipped" about a version that had never shipped, and the only escape — amend or rebase — is the operation that also defeats the leg. Two candidate fixes each looked sufficient and each was blind: counting only commits already on origin/main stops catching the coordinated edit before it merges; comparing content against origin/main stops catching an edit that has already merged. The leg needs both halves, and the way to see that was a table of three scenarios against two candidate rules, not a closer reading of either candidate.

MEDIUM — a test asserting on a configuration was satisfied by the prose describing it. The test pinning fetch-depth: 0 onto the gate's CI job grepped the workflow file for the string. The job's own comment explains why the setting is there and contains it, so deleting the setting left the test green. Caught only by mutating it. Grep a config file and you assert about a document; parse it and you assert about the configuration — the test now loads the YAML and reads jobs.template-drift.steps[checkout].with.fetch-depth.

MEDIUM — nothing pinned that the gate was wired in at all. Deleting the check.sh line and the entire CI job left all forty-odd tests in tests/test_template_drift.py green, because every one of them drives the tool directly. A test suite for a gate proves the gate works; it says nothing about whether anything runs it, and those are different claims. Two assertions now cover the second one.

MEDIUM — the hash covered bytes that do not ship, and could bake them into the ledger. It walked the working tree, so a gitignored .DS_Store in the template directory turned ./check.sh red on a clean checkout with the remedy "bump the version and archive the new files". The worse direction: present while --print-hash generated a ledger row, it was folded into the committed sha — leaving the author green and failing only on a clean CI checkout, pointing at an archive nobody had touched. The fix is ignored, not untracked, and the distinction is load-bearing: measured against a real hatchling build, a gitignored file does not reach the wheel while an untracked-but-un-ignored .orig does. Hashing git's tracked set would have hashed away a stray file that really publishes, and given a brand-new archive the digest of the empty string.

Process note — two independent skeptics disagreed, and the disagreement was the useful output. One confirmed the stray-file finding; another refuted it, having implemented and measured a tracked-set fix and shown it strictly worse. Both were right about what they tested, and neither had tested the rule that was actually adopted. A refutation kills a proposed fix, not necessarily the finding — the two have to be judged separately, and the second reviewer's evidence is what made the third option obviously correct.

T2 — Reporting template drift as a line count (20260807 22:28)

HIGH — A test that never calls the function under test cannot kill a mutant inside it. test_the_kb_identity_block_never_produces_a_hunk was written for the mutant the plan called the one that matters: rendering the old side with the KB's recorded reference and the new side with the installed one, which is what a reader of init.py:75 would naturally write, and which puts a [kb] hunk in every report on every KB. The test asserted exactly that property — by rendering both versions itself. So the mutant went into doctor._template, the test rendered its own two sides correctly, and it passed. The property was right, the altitude was wrong. It now also asserts the count doctor reports, against a synthetic pair that differs on one line: correct is two, a leaking identity block is four.

HIGH — A line-count assertion is blind to an edit that substitutes rather than adds. The invariance test edits the user's manifest and asserts the reported count does not move, which is what catches a report built from pinakes.toml instead of from two archived templates. With that defect injected, replacing fake-model with fastembed-model and final_k = 5 with final_k = 4 left the count identical — one line replaced by another is still one line on each side of a diff. The test was invariant under the implementation it existed to reject. Appending a comment line fixes it, because no substitution can absorb an added line.

Both were found by running the mutation pass, not by reading the tests. Two of five mutants survived a suite that was green, and neither survivor was a bug in the increment.

MEDIUM — A comparison that reads one of four files must not report 0. A template version denotes four consumed files; this check diffs pinakes.toml.j2. A bump touching only eval/questions.yaml renders two identical manifests, and the first implementation said 0 lines differ — true of the manifest, read as nothing changed. It is not a hypothetical: of the ten commits between the notes template's first version and its second, five touched the golden set and none touched the manifest. It now says same manifest and names what a version covers beyond it.

MEDIUM — The most-read string promised something no release can deliver. The cannot compare remedy ended "from the next template version onward the comparison is automatic". Under D-2b notes@1.0 is deliberately unarchived, so a KB recording it stays uncomparable however many versions ship after — the sentence was false for precisely the readers who see it most, which is every KB in existence. What a later version changes is the next KB, and the remedy now says that and says the missing content is gone rather than pending.

A second copy of "the variables this build supplies" was one commit from existing. tools/template_drift_gate.py leg (vi) asserts every archived version renders, under a context written out in the gate. The product now renders both sides of a comparison under template.render_context. Two literals, and the failure mode is the gate staying green while pnk doctor raises on the KB in front of it — the gate would have been asserting that the archive renders under a context nothing uses. The gate now builds its keys from template.CONTEXT_KEYS, and test_render_context_supplies_exactly_the_declared_union pins the remaining seam.

T3 — pnk upgrade, print only (20260808 00:02)

HIGH — a same-length mutant is invisible to Python's bytecode cache, so a mutation run can report the exact false negative it exists to prevent. Two of T3's eight mutants — == 1 → >= 1 on each branch of the placement predicate — came back SURVIVED. Applying one of them by hand and running the same test killed it immediately. The harness was mutating src/, and CPython invalidates a .pyc on the source's (mtime truncated to the second, size): == 1 → >= 1 changes neither, so a mutation written and reverted inside one second was executed from stale bytecode. Every mutant that did die changed the file's length.

Two things follow, and neither is about this increment.

• A mutation harness must invalidate bytecode explicitly — delete the module's __pycache__ entry after writing and after restoring. Without it, "mutation-verified" is a claim the harness cannot support for precisely the smallest, most surgical mutants, which are the ones worth running.
• The failure was silent and reported success. SURVIVED and "this code path has no test" are the same string, and only one of them was true. What caught it was disbelief — the surviving mutant looked like it should have been caught by a test that named uniqueness in its own title — not the tooling.

And the disbelief was half right, which is the second finding. Once the cache was cleared, one mutant still survived: relaxing already applied from "at exactly one position" to "somewhere" killed nothing, while the same relaxation on the clean branch was caught at once. ::test_a_hunk_whose_context_matches_twice_is_a_conflict covered one branch of a two-branch rule and read, from its name, as though it covered the rule. ::test_an_already_applied_hunk_matching_twice_is_a_conflict_too is the missing half — a user who adopted a change and kept a second copy of the block — and it was written because a mutant demanded it, not because anyone re-read the predicate.

MEDIUM — the invariance test demanded the wrong property, and passing would have been worse than failing. "A user's edit never appears in the report" was written as "the two outputs are byte-identical", and it failed: a user's provider = "fastembed" does appear, as unchanged context, because the context is what their own manifest renders to. The property that matters is narrower — their edit never appears as a +/- line — and the first version would have forced the implementation to hide context lines to satisfy a test, which is the tail wagging the dog. The distinction is now in the test's docstring: the context lines are the user's, the changed lines are the template's.

MEDIUM — textwrap broke a command across two lines, and the wrap was added for readability. The first wrapped remedy printed run `pnk at the end of one line and init` on a throwaway directory at the start of the next — a remedy naming a command nobody can copy, produced by the change meant to make remedies easier to read. _fill now glues the spaces inside a code span before wrapping. Worth remembering wherever prose meant for a terminal contains something meant to be typed.

LOW — the plan predicted a mutant would kill two tests and it killed one, because the implementation is stronger than the plan assumed. Dropping uniqueness was expected to fail both the twice-matching test and the reordered-manifest one. It fails only the first: reordering is caught by matching the hunk's lines contiguously and in order, which is a different clause. A separate mutant — every line merely has to be present somewhere — kills the reordering test. The plan's prediction assumed a looser predicate than the one built; the two tests assert two properties, which is what it was really asking for.

HIGH — the "writes nothing" test could not see a write, and its docstring stated the reason backwards. T3's central claim is that the command writes nothing, and the plan names the mutation that proves it: open the manifest for writing and confirm ::test_nothing_under_the_kb_is_written fails. It did not fail. The snapshot compared the bytes of the files only, so a rewrite of identical content was invisible, and directories were filtered out, so mkdir(".pinakes") was too. Both mutants survived and the suite was green.

Two things made it hard to see, and the second is the lesson.

• The mutation run had reported this mutant as KILLED, because the mutant I wrote also dropped a stray file — a strictly stronger mutation than the plan's. A mutant that does two things cannot tell you which one the test caught.
• The helper's docstring asserted the opposite of the truth: "Bytes rather than mtimes: an mtime comparison passes for a rewrite of identical content." Mtime is precisely what catches that; bytes are what miss it. It read as a considered trade-off, which is why nobody re-derived it. A wrong reason stated confidently is worse than no reason, because it ends the inquiry.

The snapshot now compares the path set, the bytes and st_mtime_ns, over files and directories alike, with a table naming what each of the three is blind to on its own.

HIGH — the already applied predicate asked a whole-file question about a per-hunk fact. It tested whether the hunk's removed lines occur anywhere in the manifest. A manifest is comment-dense and repeats blank lines and bare # everywhere, so any hunk deleting one could never be already applied: the user who adopted that change by hand was told conflicts — and under T4's all-or-nothing rule, that refuses their whole --apply run. It now asks whether the hunk's before image is still there, which is the same question scoped to the hunk's own region, with the pure-addition case carried by an explicit not removed guard where a reader can see it.

MEDIUM — a control test passed under the very mutant it was written to kill. The first version of "a deletion not yet applied is still clean" deleted a line in the middle of the file. With trailing context, removing a line breaks the after image's contiguity on its own, so the first half of the predicate answered and the half under test was never consulted. Only a deletion at the end of the file — where there is no trailing context — reaches it. The shape of a fixture is part of what it tests, and "it deletes a line" was not specific enough to be a test of anything.

MEDIUM — _range and _section were written by hand and asserted by nothing. Six mutants across them survived: emitting a two-number range where a one-line range is bare, dropping the empty-range back-off, an off-by-one on the start, _section returning None for everything, reading the wrong side, and dropping the column from the listing. The diff is the part of this command a user may paste into patch, and its numbers had no test. What looked like coverage was assert "[budget]" in out — satisfied by the diff body's own context line, whatever the listing said. The range check is now a property test against difflib.unified_diff rather than literals, so it cannot drift with the fixture.

MEDIUM — one message, two copies, nothing watching. cannot compare was byte-identical in doctor.py and upgrade.py, with a docstring in the second saying "deliberately the same message pnk doctor prints" — a convention with no gate, which is the shape F1 already is in this same plan. It now lives in template.py beside the archive it describes, with a test that calls both surfaces and compares.

METHOD — a scratch copy of this repo that includes .venv runs the original worktree's code. Reported by the reviewer against its own first attempt: rsyncing the tree and running pytest in the copy reported all 29 mutants surviving, including "always return CLEAN". uv run python3 -c resolved the copy's source correctly while pytest did not, so a spot-check of one mutant would not have exposed it. rm -rf .venv && uv sync --frozen --all-extras in the copy is what makes the harness honest. Together with the __pycache__ finding above, the rule is one rule: a mutation harness must prove it can kill something before its silence means anything — run a known mutant first, and treat a run with no kills as a broken harness rather than a clean bill.

MEDIUM — three of the first review's own fixes shipped with no test, in a commit whose message said each had one. The second pass reverted them one at a time and the suite stayed green: the summary line's nouns, the cannot compare: prefix on the no-template path — which docs/CLI.md publishes as a scriptable contract, "every one of them opens with cannot compare:" — and the conflict trailer that had just been rewritten to stop asserting a cause it cannot know. A fix applied under review inherits the confidence of the review and none of its scrutiny. The rule that follows: a review's fixes get the same treatment as the code they fix, mutation included.

The same pass found the wrapper's glue restore untested — dropping .replace(_GLUE, " ") printed a private-use codepoint into the middle of run `pnk init` on the one message every KB in existence receives, with 31 tests green.

MEDIUM — a test written to hold a fix could not fail. _TABLE was tightened to stop a multi-line array's continuation line being reported as a table, and the test written with it wrapped include = [ over three lines. A wrapped array of strings has continuation lines opening with ", so both the loose and the tight pattern answered [sources] and the test passed under either. It is now a unit test whose fixture is an array of arrays, which is the only shape that discriminates — and the tightening turned out to have lost [[links.kb]] and [budget] # caps, two legitimate headers, which nothing had noticed either.

MEDIUM — a new test loaded real model weights, and CI could never have told us. The test comparing pnk doctor and pnk upgrade called the whole diagnose() on a fixture naming the real sentence-transformers provider. On a checkout with pinakes[st] it downloads weights, takes three seconds and dies on a FutureWarning that filterwarnings = ["error"] turns into a failure. CI's matrix is [light], [light,pdf] and [light,pdf,claude] — it never installs [st], so this would have been green on every run and red on the machine of anyone who had the default extra the README recommends. Naming a provider nothing registers keeps the report offline and instant.

And the plan is a document a later increment builds from, so its errors are not cosmetic. The second pass found T3's own specification still carrying the whole-file predicate this increment had just proved wrong, and the exit-criteria block still instructing "compare bytes and the path set, not mtimes" — the inverted sentence whose implementation let this increment's named mutation survive. T4 reads that page next. Both now carry a dated correction rather than a silent edit, because the wrong version had already been built once from the words as written.

MEDIUM — the remedy named the newest archived version where the sentence means the oldest, and one archived version is why nobody saw it. "A KB stamped from X or later is compared automatically" printed archived[-1]. With a single archived version [-1] and [0] are the same string, so the message read correctly on every KB that exists — and would have started telling covered users they are not covered at the next template bump, on both surfaces, in the message this project calls the path 100% of KBs take. A one-element collection makes two different intentions indistinguishable, which is the same reason T1's gate leg (ii) is vacuous today and says so out loud. The test that pins it uses the fixture that archives two.

MEDIUM — the correction broke the thing it was correcting. Pass 2's ⚠ blockquote about the placement predicate was inserted between rows 1 and 2 of the predicate table. In Markdown a table block ends at the first non-row line, so rows clean and conflict rendered as literal text inside the blockquote: the rule T4 must build appeared on the published site as a one-row table. It was caught by rendering the file, not by reading the diff — and the diff looked right, because the words were all correct and in a sensible order. A correction to a document is a change to that document and needs the same reading; the fact that it is a correction is why it gets less.

LOW — three assertions compared a payload against the enum that produced it. payload["outcome"] == Outcome.NO_BASELINE.value is green under any rename of both sides, which is exactly the rename that breaks a consumer's parser. Eight JSON strings lived in that gap. The replacement writes the keys and values out as literals, once, so a wire change is a visible diff in a test rather than a silent break in somebody's script. A test that derives its expectation from the code under test is a tautology wearing an assertion's clothes — the same shape as the "[budget]" in out assertion two passes earlier, which the diff body satisfied.

HIGH — three edit batches aborted partway and I reported all three as landed. The fixes were applied by scripts that walked a list of (file, old, new) and wrote each file as they went. When an anchor failed to match — because an earlier edit in the same run had already changed the text — the script raised, leaving the files before the failure written and the files after it untouched. Six edits were lost that way: the assertion pinning the oldest-archived-version fix, the up to date branch assertions, three of the four cannot compare: prefix assertions, a docs/KB-UPDATES.md contradiction and a docs/RETROSPECTIVES.md heading. Two of them were the headline fixes of the commit that claimed them, and the commit message, the retro fragment and docs/VERIFICATION.md all said they were pinned by a named test. The next pass found the test asserted nothing of the kind.

Three rules came out of it, and the third is the one that would have caught this on its own.

• An edit batch is all-or-nothing. Resolve every anchor first, write nothing until all of them match, and name the file and the anchor when one does not.
• Never let one batch depend on text an earlier edit in the same batch produced.
• A claim that a fix is pinned is a claim about a failing test. Revert the fix and watch the suite go red, or do not write the word "pinned". Four separate fixes across three passes were described as pinned by tests that were green without them — and each was found by the next reviewer doing the revert I had not.

MEDIUM — an assertion can be true because the current wording is kind. "pnk init appears unbroken in the output" passed with the code-span protection removed, because at this width the remedy's spans happen not to straddle the wrap column. It was a real assertion of a real property that simply could not fail on this input. The property belongs to the wrapper, so it is now tested against the wrapper, with a fixture built so that plain textwrap provably splits the span — and a second assertion that checks the fixture still has that property, because otherwise the test degrades into the one it replaced the moment someone edits the string.

Where the review loop stopped, and what each pass cost. Five adversarial passes, finding 30 → 22 → 13 → 6 → 1. The fifth found no defect in shipped behaviour — one documentation overstatement and seven coverage gaps in code that was correct — which is the signal the loop was waiting for. What the shape of that curve says is worth more than any single finding: the first pass over an increment is not the expensive one. Passes 2 and 3 each found that a previous pass's fixes were wrong or untested, and pass 4 found the tooling that had lost them. A review that stops at one pass stops before it has reviewed anything it changed.

The classes, in the order they cost the most:

Class	Instances	Why it survives a careful reading
A fix claimed as "pinned" whose test is green without it	4	The claim is about a failing test, and nobody runs the failure
A test that cannot fail on its own fixture	3	It asserts a real property of a real surface; only the input is wrong
A correction that broke what it corrected	2	The words are right, so the diff reads right
A whole-file question standing in for a per-hunk one	1	Both are true on the fixture that motivated the feature

T4 — pnk upgrade --apply (20260808 05:14)

MEDIUM — a plan can specify a test that cannot discriminate, and the name is what hides it. T4's plan named ::test_a_key_carried_only_by_a_conflicting_hunk_is_not_recommended as the load-bearing operand test: the one that separates parse(base + applied) from parse(ours). It cannot. Under the same plan's all-or-nothing rule a conflicting run refuses, writes nothing and prints no recommendation at all — so both operand choices produce an identical observable, and the test would have passed under the wrong implementation while reading as the proof that it was right. The case that actually discriminates is an already applied hunk: skipped, its key already in the file, and therefore not something this run introduced. Written as ::test_a_key_carried_only_by_a_skipped_hunk_is_not_recommended. The plan's own ground rules say every test name in it is a prediction and not binding; what this adds is that the scenario can be a prediction too, and a scenario nobody re-derives is a test that measures nothing under a name that says otherwise.

MEDIUM — the consent predicate had a fourth near-miss the plan did not enumerate. D-10 requirement 2 lists three cases where a spending-cap heading must not appear — an already-applied budget hunk, a conflicting run, and a bump touching no [budget] line — and specifies the predicate positionally: a clean hunk falls inside [budget]. The shipped template's only real [budget] drift (M3) rewrote three comment lines as well as two caps, so a hunk that moves only comments is inside [budget], applies cleanly, and moves no money. Positionally it would print a spending-cap heading with nothing under it — the fourth near-miss, and the one most likely to recur, because comment churn is what template drift mostly is. The predicate shipped as a clean hunk inside [budget] that changes at least one key, still structural and still with no key-name list, plus its own negative control. The plan's requirement 2 argues for exactly this and its three-case list is what fell short of its own reasoning.

LOW — an open correction that names a fork resolves to both, and saying which matters. The CRLF item required T4 to "either preserve each line's ending or refuse a manifest whose endings are not uniform, and say which". The answer is both, and the split is not a hedge: a uniform CRLF file is preserved, because that is an ordinary Windows manifest and rewriting it would be a change nobody asked for; a mixed file is refused, because it is already the product of two tools disagreeing and picking a winner silently rewrites lines the user never touched. The report path is unaffected either way, since reporting reads.

LOW — a refusal ordering decision that only shows up on the second run. Content refusals (a conflict) are checked before environmental ones (a stray .orig, a held lock). The other order is defensible until you follow it through: it tells a user to clear a backup file so that the retry can tell them their manifest conflicts anyway. Two round-trips to deliver one permanent fact.

The mutation record, because "pinned by test X" is a claim about a failing test. Eleven mutants run 20260808 05:14; each is the code broken on purpose and the tests that noticed, and two of them are here because the first attempt measured nothing:

Mutant	What went red	The control that had to stay green
remove the conflict refusal	…refuses_entirely_when_any_hunk_conflicts (+2)	—
write the .orig before the conflict check	the same test, on assert not _backup(root).exists() — checked by reading the assertion that fired, not the test name	it must not fail on byte-identity instead, or the ordering is not what is pinned
remove the re-parse	…unloadable_manifest_is_rolled_back, alone	—
drop the rebuild-naming line	…names_the_rebuild…, alone	…does_not_run_a_sync stayed green — that pair is the whole point of finding it
a key-level writer instead of the hunk applier	…the_comment_the_template_added_is_present_after_apply (+2)	—
suppress the spending-cap heading	both-values ×2, ordering	…budget_hunk_is_applied_like_any_other_hunk stayed green; if it goes red the heading tests are entangled with the write
print the heading unconditionally	all four negative controls (+4)	—
print the new value only	both-values ×2, on the old value's absence	—
write before any printing	ordering	both-values stayed green — which is exactly why ordering needs its own test
write requires_pinakes	…is_never_written (+2)	—
recommendation over every hunk, not the applied ones	…carried_only_by_a_skipped_hunk…, alone	the other three requires_pinakes tests pass under the wrong operands, which is what makes that one load-bearing

Two of the eleven were measured wrong on the first attempt, and both failures looked like success. The syntactically-invalid mutant made pytest print ERROR, not FAILED, and a harness counting FAILED lines read that as nothing noticed — the mutant survives: a false negative that would have sent someone to strengthen a test that was already correct. And the .orig ordering mutant turns twenty-two tests red, so the run "failing the right test" says nothing until you read which assertion fired. A mutation harness needs its own controls for the same reason the tests do.

MEDIUM — the first adversarial pass found five defects, and four of them were invisible from the tests because every one of them is a property of the write, not of the diff. T4's whole test surface was built from a plan concerned with placement, consent and refusal; none of those notice what happens to the file's inode.

Found	Why it matters
str.splitlines() breaks on \u2028, \u2029 and \x85; split("\n") does not	the report and the writer would disagree about which lines the file has, so a hunk the report called unique could match somewhere else, or nowhere. All three are non-ascii under TOML's own comment grammar, so a manifest can carry one and still load. Refused, because rejoining on \n would silently turn it into a newline in a file the user owns
os.replace onto a symlinked pinakes.toml destroys the link	the real manifest is left elsewhere holding the old text, and the KB now has a regular file. sidecar.write had already learned this and resolves the same way — the lesson was in the repository and the new writer did not inherit it, which is the argument for the write helper being shared rather than re-written
mkstemp creates 0600, so the rename narrowed the manifest's permissions	a user who had made it group-readable loses that, silently, on an unrelated command
a dotted key parsed to its first segment	budget.monthly_eur = 30.00 would be announced as a key called budget — in the consent line, which is the one output that must name what it is asking about
the rollback wrote non-atomically	it restores the file after a failed write; doing that non-atomically reintroduces the failure mode it exists to recover from

All four fixes were then re-mutated, and each turns exactly one test red. The fifth has no test and is not claimed to have one: a crash mid-restore cannot be staged in-process, and a row in docs/VERIFICATION.md says none rather than pointing at a test that would pass either way.

HIGH — the second pass found the one defect that would have printed a false spending-cap heading, and it was a correct function used one scope too wide. Hunk.section is read out of base by scanning backwards from the hunk's first changed line, so it is the table the hunk starts in. That is right for the placement listing, which labels a region. It is wrong for attributing a key, because a hunk that adds a whole new table carries keys belonging to a table base does not contain — and every one of them was credited to the preceding table. A template adding a table directly after [budget] would have had each of its keys announced under a spending cap changes, and one added after [chunking] would have invented an index rebuild.

Two things are worth keeping about how it was missed. It is not a bug in _section; it is a value being reused at a scope where its definition no longer holds, which no test of _section could ever catch. And every consent test passed, because each fixture's hunk stays inside one table — the negative controls guard against a heading printed when nothing moved, and this prints a heading naming the wrong thing, which is a different failure they were never shaped to see.

LOW — two refusals in splices were unreachable from every fixture, so they shipped unseen. One is unreachable by construction (_placement established uniqueness over the same text moments earlier) and the other needs a manifest repeating a region in a shape difflib still calls two hunks. The function is public now, for the reason fill and restamp are, and both guards have a test that reverting them turns red — including a control on the overlap guard, since a refusal that fires on everything passes the same assertion as one that fires correctly.

MEDIUM — and committed in this increment, in the file that documents the mistake. Pass 2 was committed with ./check.sh 2>&1 | tail -2 && git commit. In a pipeline the shell reports the last command's status, so tail succeeding made a failing ruff invisible and the commit landed over a red gate. check.sh's own header comment describes this exact shape as the reason the script exists — "uv run pyright | tail -1 && git commit reports the tail exit status, so a failing checker looks green" — and it was reproduced by a reader of that comment, one line below it, while running that script. A gate is only a gate when its status is the one the next command reads: write ./check.sh > log 2>&1; echo $?, never ./check.sh | tail.

LOW — pass 3, and both findings are the first two fixes not being finished. Writing through a symlink (pass 1) puts the backup beside the file it backs up, in another directory — and the output still printed its bare filename, which is true of every ordinary KB and misleading in the one case where finding the file takes work. A fix that changes where something lands has to be followed into whatever names it. And docs/CLI.md's refusal row still listed five causes when the code had eight, because passes 1 and 2 each added a refusal and neither went back to the table that enumerates them.

The same-manifest gap is stated rather than closed, deliberately. A version bump that leaves the manifest byte-identical produces no hunks, so --apply writes nothing — the [kb] template restamp included — and that KB keeps reporting drift with no way to record the new reference. T4 specifies --apply in terms of hunks and there are none, so writing anyway would be behaviour the plan does not describe. It is a test asserting today's behaviour, a row in docs/VERIFICATION.md, and an item proposed for plans/20260731_1202-open-corrections.md — not a decision taken quietly inside an increment.

MEDIUM — pass 4 found nothing in the code and three falsehoods in documents the increment never opened. docs/README.md still said "what is still a proposal is --apply"; docs/KB-UPDATES.md §1 said adoption "is still deferred" and its drift-axis table said "Adopting it is still absent: nothing writes the change into a user's manifest". All three were true when written and false the moment T4 compiled, and none is in any file T4 touched — which is exactly why the audit-the-neighbourhood convention exists and why a diff-scoped review cannot satisfy it. The search that found them was two greps over docs/ for the claim being falsified ("writes nothing", "--apply" near planned/deferred/absent), not a re-read of the change.

And the counterpart worth stating: docs/ROADMAP.md's ## 0.19.0 section still says "It writes nothing" and was left alone. That section is a record of what 0.19.0 shipped, and it was true of 0.19.0. A release-notes entry rewritten to match a later release stops being a record. The line that does need the sweep is the page's own "Where things stand right now", which is current-state and is the release's job, not the increment's.

Every documented exit code was then run rather than read — report 0, --apply 0, a conflicting report 0, a conflicting --apply 1, cannot compare under --apply 3 against the shipped notes, and an up-to-date --apply 0. Six rows, six commands, matching the table in docs/CLI.md.

Pass 5 found no new defect and is recorded anyway, because what it did is the part usually skipped. It checked the two factual claims the increment writes into three documents and one runtime message — that upgrade.apply is the only thing rewriting a pinakes.toml after pnk init (init.py:88 is the other write; nothing else in src/ writes it), and that init's .gitignore covers .pinakes/ and nothing else. Both hold. Then it mutated the seven T4 tests no earlier mutant had killed. Every T4 test is now killed by at least one mutant — which is a weaker claim than "every test asserts its property", and it is the strongest one a suite can actually make about itself.

T5 — The plan asked for a decision the file next to the code had already taken (20260808 06:28)

HIGH — D-4 was open in the plan and settled in manifest.py, three lines below the tuple it was about. The plan spent a four-option table, a recommendation and a paragraph of D-12 cross-reference on "what happens to vector_tier = "sqlite-vec" before the tier exists", and framed the choice as turning on a judgement about docs/MANIFEST.md: is its row a promise or a disclosure? Meanwhile GRAPH_CHANNELS' docstring, at manifest.py:52 against VECTOR_TIERS at :51, already stated the answer as a rule — "a manifest that can ask for a mode the code does not implement is a manifest whose setting silently does nothing, and table.choice refusing the name is how a user finds that out at load time" — and applied it to "ppr", a value in the very next row of the same documentation table. The plan cites neither.

What that changes about reading a plan. A plan's decision table is a list of questions its author could not answer from the plan, which is not the same as questions the repository has not answered. Two of the plan's four open recommendations here were about consistency with existing behaviour, and the cheapest evidence for both was adjacent to the line being changed. The habit worth keeping: before weighing a plan's options, look at what the sibling key does — the file is often more decided than the document about it.

MEDIUM — the plan's own two halves disagreed about what T5 could deliver. It asked for resolve_tier to be called by both sync and search, "so meta's claim and the code path cannot disagree", and then admitted two paragraphs later, correctly, that "with exactly one real tier there is nothing else to discriminate". Both cannot hold: if there is nothing to discriminate, search has no dispatch to make, and a tier parameter threaded into _vector that can hold one value behind an unreachable branch buys a shape that looks like a shared decision while being decoration. Built the resolver with one caller and said so in its docstring. This is the eighth of the template-release plan's own measurements or specs to be wrong, and the second found by building rather than by reading it.

A smaller one, on honesty in a one-tier world. The first draft of resolve_tier was return "numpy" — correct, and a function that ignores its only argument. It became return "numpy" if tier == "auto" else tier, which reads the manifest and honours an explicit tier. Today both arms return the same string, so no test can tell them apart; what the second form buys is that the increment restoring "sqlite-vec" gets it honoured by that line and owes only auto's side of the choice. Worth the branch; not worth pretending a test covers it.

T7 — pnk templates, and a template declares its files (20260808 09:54)

HIGH — a test with two cases in one body only holds the case that runs first. test_a_template_file_entry_that_escapes_the_target_is_refused covered both write-side escapes: ../../evil.md and a symlinked directory in the target. It was green, and the mutation pass showed it was green for the wrong reason. Removing the destination containment check turned it red on its first assertion — ../../evil.md escapes the template as well as the target, so the other layer caught it with a different message — and pytest stopped there. The symlink case, the one only the removed check can catch, never executed under the mutation it existed to detect. Split into two tests; both now go red, and the source-side test stays green. The general shape: when two guards can both catch one input, a test that exercises that input proves nothing about either of them, and a shared test body hides which one fired. This is the same family as T4's …by_a_conflicting_hunk_… — a test that cannot discriminate — but arrived from the opposite direction: not a case that looks identical under both implementations, but a case caught twice.

HIGH — a plan's rule sentence and its test list can specify different things, and the test list was right. T7 states that each files entry "is validated to land inside the target KB", then lists a test refusing "a symlinked directory inside the template tree". Those are two layers: a symlink in the template resolves outside the template while its destination stays inside the target, so the target rule cannot catch it and the named test could never have passed. Both are built. The write side stops an entry writing into a directory the user never pointed Pinakes at; the read side stops it copying something the template does not own into a KB that is then committed and published — which for a public repo is the more expensive direction. Read a plan's test list as part of its specification, not as an illustration of it. Where the two disagree, the test list has been forced to be concrete and the sentence has not.

MEDIUM — a coverage row that cannot fail is worse than no row. T7's free-path deliverable asked for main(["templates"]) and a module added to test_paid_path.py's surface list. But pnk init already runs in the same fixture, so pinakes.template is in the import graph regardless, and run_templates lives in pinakes.cli — no row would have failed if the new call were deleted. The call was added and the row was not, with the reason written at the call site so the omission is not "fixed" later. Eight of this plan's measurements were already wrong; this is the ninth and the tenth, and both were found by trying to satisfy its own test list rather than by reading its prose.

MEDIUM — a new listing command inherits every failure mode of the thing it lists, multiplied. template.describe raising a bare OSError on a damaged install is an open correction from T3, and it reaches init, doctor and upgrade. Before pnk templates, that cost you the run that named the damaged template. A listing iterates all of them, so one bad directory produced a traceback and reported nothing about the templates that read perfectly. The general fix stays open — it is not this increment's to settle — but the blast radius this command introduced is contained here: the failure is caught per template, shown as a row, and the exit code is non-zero. Adding a "list everything" surface converts a per-item defect into a whole-command defect, and that conversion is the new increment's to own even when the defect is not.

LOW — the runtime agreed with a bug the linter caught. A missing Callable import passed the whole test run and failed ruff. The worktree's uv environment is Python 3.14, where PEP 649 defers annotation evaluation, so an annotation naming an undefined type never evaluates. On the 3.13 this project requires, it is a NameError at import. A green test run on a newer interpreter is not evidence about the one in requires-python — which is what the linter is for, and why ./check.sh runs it before pytest rather than after.

HIGH — a new key can defeat an existing invariant by living in the file that invariant excludes. tools/template_drift_gate.py hashes every file in a template except template.toml, and that exclusion is load-bearing: hashing the file that declares the version would make every bump change the hash by construction, so "a version bumped with no content change" could never be detected. The gate's own limit (b) recorded the cost as the description could be edited unnoticed — true, and cheap, while a description was the only editable thing in there. T7 added files = [...] to that same file, and files decides which files a KB is stamped with. So the key that determines a template's stamped content landed in the one file excluded from the check that exists to stop stamped content changing without a version bump. Closed by folding only the list into the digest: name, version and description stay out, so leg (ii) can still fail. An absent key contributes nothing, which is what makes the change backward-compatible — the shipped notes hash is byte-identical before and after, so no published _versions.toml row needed migrating. When an increment adds a key, ask which existing invariants were scoped by the assumption that the file holding it did not matter. Nothing in T7's plan connected the two; the link is only visible from the gate's side.

MEDIUM — git checkout -- <file> restores to the last commit, not to the state before the mutation. The mutation pass ran fine against template.py, whose changes were already committed, then silently destroyed the uncommitted drift-gate work when the same restore step ran against it. The mutation's result was valid — both target tests went red, the negative control stayed green — but the code came back as main's, and the next full run showed two failures that looked like the fix was wrong when it was simply gone. Commit before mutating, always: the restore step is only as precise as the thing it restores to, and a mutation harness that quietly reverts real work reports a false failure rather than a false pass, which is the better direction but still costs a debugging cycle.

Open corrections 1 and 3 — a guard can route a failure into a wrong message (20260810 01:29)

HIGH — turning a traceback into a PinakesError moved the failure into a handler that already existed and said the opposite thing. Item 3's whole content is stop these five reads escaping as tracebacks. Doing it made describe raise TemplateError — and doctor.py and upgrade.py both already wrapped their describe call in except PinakesError, answering it with "is not installed here" and a remedy about installing the template. That arm was correct while the only thing reaching it was a template genuinely absent; a damaged install had been going straight past it as a traceback. So the fix silently recruited a handler written for the opposite case, and a user whose template.toml was unreadable would have been told to install a template sitting right there.

A traceback is loud and a wrong sentence is quiet, so this is a downgrade that reads as an upgrade. Both surfaces reported WARN, both exited 3, every test stayed green — the increment's own tests included, because they assert the new messages and the pre-existing ones assert the absent-template case that still works. Nothing was red. It was found by asking who else catches what this function now raises, which is a different question from does my change work.

The correction is TemplateNotInstalledError, a subclass so that every existing except TemplateError keeps working, with _unknown as its only raiser and the two callers splitting the arms. Both surfaces get their own test, because the wording is a fact with one home but the routing is a decision each caller takes for itself — one test would leave the other free to merge the cases back.

Generalises past exceptions: widening a type is an interface change on every except upstream. The grep that finds it is not for the function being changed but for the type it starts raising, and the question is whether each catcher's answer is still true for the new cause. PinakesError is caught in 30-odd places here precisely because it is the type that means print this and stop, so anything newly raised as one inherits whatever those handlers already say.

Second, smaller, and mechanical: git checkout -- <file> during a mutation pass deletes the fix being verified. The pass ran before the commit, so restoring after each mutation restored to main — three mutations in, both source files were back to their unfixed state and the tests were "failing correctly" against no fix at all. The evidence was still true and the work had to be typed again. Commit, then mutate: BUILDING.md orders the steps that way, and this is the reason rather than a convention.

A release sweep is table-shaped, and a narrative is not a row — 20260811 12:18 UTC

What happened. With every plan built out and the open-corrections list empty, the first check of the next session was what does the repo say about itself. docs/ROADMAP.md said 0.21.0. Its release table carried a 0.22.0 row, its per-release section carried the full 0.22.0 write-up, and its open-corrections section said none live, all four shipped in 0.22.0 — but the two prose blocks that state the project's position, ## Where things stand right now (stamped 20260808 06:41) and ## The template release, were three releases behind. One of them still said "T4 and T7 are still to come" about increments that had shipped on 20260808 and 20260808.

Why it survived five sweeps. The landing checklist asks which file to edit, and the file was edited — every time. What a sweep naturally finds is the row it is adding: a table has one line per release, so the release being cut points straight at its own row. A paragraph that summarises all releases has no row to add, so nothing in the act of cutting a release makes it obvious. The per-release sections and the tables were correct at every commit; only the summaries were wrong — which is the worst arrangement, because a reader checking one against the other finds agreement in five places out of six.

The second instance is sharper. docs/README.md's plan-routing table — the table whose whole job is to tell a session which plan is live — had no row at all for plans/20260811_0720-decisions-gates-and-corrections.md, the plan CLAUDE.md names as the live build order and the authority for eight decisions. The plan was written, its six increments were built and landed, and the index of plans never learned it existed. A missing row is invisible to every check that reads rows; only asking "is everything that exists listed here?" finds it, and that question is not part of landing an increment.

What generalises. "Update the doc" and "update the doc's summary of itself" are different actions, and only the first is prompted by the work. A document that both enumerates and summarises will drift at the summary, in the direction of the last release that bothered to rewrite prose. Two checks are worth adding to a landing:

• Grep the docs for the previous version number after a release, not just for the fields the checklist names. grep -rn "0\.21\.0" docs/ would have found both ROADMAP blocks in seconds.
• Ask what is missing, not only what is wrong. The routing table's defect had no wrong text to find. Reading ls plans/ against the table is a ten-second check that no diff review performs, because nothing in a diff is absent.

And the check found more than it was looking for. Auditing the neighbourhood surfaced that the 20260807 audit's 40 documentation corrections have never been worked — the file has one commit, the one that created it — and that the same audit deferred a full review of docs/ROADMAP.md until after T2, which shipped in 0.18.0. Neither is visible from any release's own sweep either.

A row can be complete, correct, and in the wrong place (20260811 13:27)

HIGH — five release rows were out of order in three sequences, and nothing could see it. docs/ROADMAP.md's release table and its per-release sections both read 0.20.0, 0.22.0, 0.22.1, 0.21.1, 0.21.0, 0.20.1; docs/STATUS.md's roadmap table put 0.15.1 after 0.16.0 and 0.20.1 after 0.22.1. Every misplacement is wrong on both readings — SemVer and release time — so no ordering convention made them right. CHANGELOG.md was checked and is clean, headings and link definitions both.

Ordering is a property of the sequence, not of any row in it. That is why nothing caught it: the tables are complete, every anchor link resolves, mkdocs build --strict is green, and a reader checking any single row finds it correct. Every check this project owns reads rows.

How it happened, recovered from the six release commits. 0.20.1 was appended correctly (2da0e07). 0.21.0 (96b3b35) then inserted its section one position too early — after 0.20.0's rather than after 0.20.1's — and the next three sweeps (c83e877, df832fe, 93c20ab) each used that same slot.

Commit	Release	Section tail after it
d429a2c	0.20.0	… 0.19.0 0.20.0
2da0e07	0.20.1	… 0.20.0 0.20.1 ✅
96b3b35	0.21.0	… 0.20.0 0.21.0 0.20.1 ← the first error
c83e877	0.21.1	… 0.20.0 0.21.1 0.21.0 0.20.1
df832fe	0.22.0	… 0.20.0 0.22.0 0.21.1 0.21.0 0.20.1
93c20ab	0.22.1	… 0.20.0 0.22.1 0.22.0 0.21.1 0.21.0 0.20.1

The tail was locally self-consistent at every step, which is the whole lesson. After the first error it read strictly newest-first, so each following sweep saw a coherent pattern around its own edit and matched it — reproducing the error is what reading the neighbourhood carefully produced here. Only the join between the ascending head and the descending tail was wrong, and that join is a single line no sweep's diff ever touched. Read the sequence, not the neighbourhood.

It had already been found and left. The 20260807 documentation audit's finding docs/STATUS.md:303 names the 0.15.1 row exactly, with PyPI upload times as evidence. It sat unworked for four days, and in that window three more sweeps added three more misplaced rows. A verified finding nobody schedules is worth what an unverified one is.

The fix is a gate, not a checklist item — tools/release_order_gate.py, in check.sh and CI, on the threshold this project already applies (status_header_gate.py: a checklist missed it four times). Two design points are the reason it will still work in a year:

• Direction is declared per sequence, never inferred. Inferring it from whichever way most adjacent pairs agree would let a badly scrambled file elect its own answer and pass.
• A sequence shorter than a floor is a failure, not a pass. An empty sequence is sorted by definition, so a pattern that silently stops matching — a reformatted table, a changed heading style — is exactly how this class of gate dies quietly. A test asserts the real documents clear that floor, so a reformat goes red in the commit that does it.

Watched failing before it was trusted: run against the pre-fix tree it names all nine misorderings and exits 1; CI scrambles a copy and asserts both the exit and the stated reason.

E1 — pnk ask, and the output nobody could check (20260811 15:17)

MEDIUM — a doc that pastes command output goes stale silently, and nothing in this repo can see it. docs/GUIDE.md § Searching showed a pnk search result whose last two lines were the escalation notice — Paid synthesis (pnk ask --deep) is planned for the deep release. E1 changed that sentence in cli.py, and the pasted block kept printing the old one. It would have shipped: ./check.sh is green on it, mkdocs build --strict is green on it, and every link in the file resolves. The conventions already say verify a doc by running the commands it shows (docs/README.md) — this is the first recorded instance of the rule catching something, and it caught it because the increment happened to edit the very sentence, not because anything checked.

What makes this class different from a stale sentence. Prose describing behaviour is written by someone who read the code; pasted output is a measurement, and a measurement has no author to notice it aged. There are three such blocks in GUIDE.md and two in CLI.md. Nothing pairs any of them with the code that produces it.

Not turned into a gate here, and the reason is worth recording. A gate would have to run each documented command against a fixture KB and diff the output — which is a golden-output test on formatting, the most brittle kind, and it would fail on every deliberate wording change with no way to tell those from regressions. What is cheap and was done instead: both new blocks in this increment were pasted from a real run (tests/demo-kb for the calibrated one, tests/partner-kb for the uncalibrated one), so at least the version now on main was true once, with the KB that produced it named beside it.

LOW — a test that can only fail in a later increment must say so in its own docstring. test_a_confident_kb_gets_cited_evidence_and_the_price_of_one_call asserts no ledger.jsonl exists after pnk ask. Nothing in this build can spend, so the assertion cannot fail today; it is a tripwire for E4, which adds a paid loop to this same command through the shared _retrieve. Left unexplained it reads as dead weight and the next reader deletes it — one increment before the one it was laid for.

LOW — the vocabulary was spelled twice in one file. run_search compared confidence against the string literals ("low", "unknown") while _escalation, forty lines below, compared against search.py's LOW/HIGH/MEDIUM constants. Both correct today. The renderer also asked if escalation.branch != "none" to decide whether to name the deep release, putting the branch vocabulary in a third place — and the decision that actually mattered (never offer to answer a question nothing matched) lived in none of them. Both fixed: the constants throughout, and the notice carried on the value rather than re-derived at the print site.

E2 — The deep release's round estimator (20260811 16:15)

HIGH — the worst case left out the one input the user controls: the question itself. The plan's formula prices carried memory, final_k passages and a prompt constant, and the first draft implemented exactly that, with the question folded into PROMPT_TOKENS. But a question arrives as an argv string — pnk ask "<question>" — with no length limit anywhere in the CLI, and it is carried into every call of a run, so a 50,000-token question would have been reserved for at 1,500 tokens. Fixed by pricing it separately (QUESTION_TOKENS) against a stated ceiling (QUESTION_CHAR_CEILING) that E4 must enforce, since this module refuses nothing but a stale price table and an oversized request. Lesson: when a worked example pins an estimate to the cent, it also silently fixes the size of every input the example did not vary. The question was the one term with no bound anywhere in the system, and the arithmetic looked right because the example never made it big. Ask what is in a call's input that the formula does not name — not whether the formula was implemented correctly.

HIGH — the second review pass found a ceiling below its own measurement, in a module written to refuse exactly that. PASSAGE_ENVELOPE_TOKENS carried the comment "the longest path — heading_path pair is under 120 characters", asserted without running anything, and 100 tokens was sized from it. Running it — by extending tools/measure_passage_tokens.py to report the envelope as well as the chunk — returned 220 characters, about 110 vendor tokens at the pessimistic conversion the module's other constants use. The constant was under-reserving against a number nobody had taken; it is now 250, with the measurement and its date beside it. Lesson: the review question that found it was "which of these numbers did I measure, and which did I merely write?" — asked of a file whose every other constant carries a command. A measured neighbour makes an asserted one look measured too, and prose like "measured from the corpora above" is how the two become indistinguishable.

MEDIUM — a total divided by its call count and multiplied back was not the total. Decimal division is exact to 28 significant digits and no further, so per_call_eur * calls landed one digit above total_eur at the shipped defaults — meaning, in the other direction, a per-call reservation could sum to less than the operation it belongs to. Caught by a test asserting the two were equal. Fixed by deriving the total from the per-call price rather than the per-call price from the total, which removes the class rather than the instance: no set of constants can reintroduce it. Lesson: Decimal fixes base-10 representation, not associativity — money code should multiply a unit price up, never divide a total down, and the direction of the last digit is chosen by nothing.

MEDIUM — a "measured" docstring falsified by the fix in its own increment. The docstring above cited the two 28-digit values it was measured at; the fix changed the arithmetic that produced them, so re-running the same comparison now round-trips exactly and the cited numbers describe code that no longer exists. Rewritten to say what was observed and when it stopped being reproducible, rather than left reading as a claim about the shipped module. Lesson: the measurement most likely to go stale in a docstring is the one that motivated the change in the same diff — grep your own increment's fixed defects for the numbers that justified them.

LOW — a purity test written as a denylist, in a repository that had already learned not to. The first version asserted deep/estimate.py imports no anthropic, sqlite3, httpx, requests — a list that is never finished, and whose next omission is always the one that matters. check.sh's NUL scan records the same lesson about file suffixes ("a denylist of binary formats is never finished"), one file away. Rewritten as an allowlist of what the module may import, so a new import has to be argued for in the test. Lesson: for "this module stays pure", enumerate what is allowed. The repository's existing gates are worth reading for the shape of the assertion, not only for the rule.

A finding handed to E4 rather than solved here. At the shipped defaults (final_k = 8, [chunking] max_tokens = 510, [budget] per_operation_eur = 0.30) the cheap branch prices at EUR 0.2627 and fits inside the cap; a five-round loop prices at EUR 2.81 and is 9.4x it. So on a stock KB, pnk ask --deep would answer a confident question and refuse an uncalibrated one at round 0 — which is precisely the combination D-22 option E was chosen to avoid, arrived at through the caps instead of through the signal. E2 declines to fix it by lowering a ceiling: that is the trade PAGE_TOKEN_CEILING refused, and the numbers are conservative by design until E6 measures them. It is pinned as tests/test_deep_estimate.py::test_the_shipped_defaults_leave_the_loop_outside_the_default_operation_cap, reading both sides out of the manifest defaults so it tracks them rather than restating them.

E3 — The deep client, and four rules that were about to be copied (20260811 17:12)

HIGH — a second paid entry point is where a "one home" rule quietly becomes two. E3's section of the plan says the key is PINAKES_ANTHROPIC_API_KEY and never an ANTHROPIC_API_KEY fallback, "enforced the same way here" — a sentence that reads equally as reuse it and as write it again. Writing it again would have been the smaller diff, and it would have produced two copies of four rules that each fail silently when they drift: the key's name, max_retries=0, whether a failed call billed, and how a reconciliation is computed. CLAUDE.md calls an ANTHROPIC_API_KEY fallback "the same defect, one layer apart"; a second copy of the rule forbidding it is the same defect one file apart. So src/pinakes/paid.py exists, holding all four with no client in it, and extract/claude.py now imports what it used to declare. What made the move safe rather than brave was that its 1,606-line suite is unchanged: the only test that failed was the allowlist count, which is the gate refusing the widening until it was declared.

The module is deliberately not allowlisted, and that is the interesting part: classify is handed the caller's already-imported SDK module rather than importing one, so gate 2 scans it like any other file and would refuse an import anthropic added to it. The allowlist stays two entries — extract/claude.py and deep/client.py — which is the complete list DESIGN §1 describes.

HIGH — the shared classifier had no direct test, and it decides void versus unknown outcome. Moving it surfaced that every branch of _classify had only ever been reached through a fixture that raised an already classified error: the replayer constructs TransportError itself, so the mapping from an SDK exception to a billability had never been executed by a test in either suite. The relationship it turns on is the one stubs/anthropic.pyi warns is easy to get wrong from memory — APITimeoutError is a subclass of APIConnectionError, so checking the parent first classifies every timeout as not-billed and voids a reservation for a call that may have been charged. It now has five tests against a fake SDK whose hierarchy is held against the stub's, and against the real package where it happens to be installed.

MEDIUM — the renderer was spending 90% of a ceiling on an identifier the model never emits. The first draft of render_passages mirrored cli.py's printed block exactly, citation line included — which carries the path and the heading a second time. E2 measured PASSAGE_ENVELOPE_TOKENS against one copy (220 characters, the widest real passage in the corpora) and set it at 250 with 2.3x headroom; two copies came to 452 characters, 226 of the 250 reserved tokens, leaving nothing for a KB whose headings run deeper than the two corpora measured. Found by writing the test that pins the arithmetic, not by reading the code. The fix was to drop the citation line rather than raise the constant — E2's own rule, and PAGE_TOKEN_CEILING's: a ceiling that close to its own measurement is not a ceiling. The test now asserts the envelope stays under half the constant, because "inside it" was true of the draft too.

MEDIUM — three smaller findings from the same pass, each a disagreement between two halves of one bound. The subproblem cap was computed twice, once for the request's schema and once for the parser that re-checks the response — the shape where the second check silently becomes about a different limit than the first, at which point one of them is decoration (now subproblem_cap, one function, driven through both users by one test). answer_schema wrote max(passages, 1), so a call over zero passages would have declared citation [1] legal to the API and had every index refused by the parser — prose with nothing behind it, paid for; both halves now refuse (NoEvidenceError), which is E2's reason for not pricing the none branch, arriving one layer down. And billed_call voided on any exception the transport did not classify — but a defect is not proof the call never billed, and proof is what INVARIANTS requires before a void, so an unclassified failure is now left unresolved instead.

LOW — a test that asserted its own base class. The first version of "every error carries a remedy" checked issubclass(..., PinakesError), which is true by construction and would have passed for a subclass that omitted the keyword entirely. It reads the super().__init__ call out of the parsed source now. Worth recording because it is the third time in this repo a safety check has been written in the shape that cannot fail; the tell each time was that it passed the moment it was written.

What E4 inherits from this increment, beyond what the plan already names: the three bounds E2 prices against are enforced in the client — the question ceiling, the carried-memory ceiling (CARRIED_MEMORY_CHAR_CEILING, added beside the token constant it derives from), and final_k passages per answering call — and each refuses rather than trimming. E4 should refuse earlier and more kindly, with a sentence about the command; what it must not do is assume the client will quietly cope. Citations arrive as passage numbers, so E4 owns the mapping back to documents, and E7's rule that a suggestion's endpoints must be documents this run retrieved is already a property of the wire format rather than a check to add.

Second review pass — the knob nobody would have passed, and a version nothing checked. Two more, both found by reading the module as a stranger would rather than as its author:

• max_tokens was a parameter on both request builders, defaulting to the estimator's constant. Harmless as written and a hole by construction: output bills at five times the input rate and is two thirds of a round's whole price, so a caller passing a larger ceiling is billed for output no reservation covered — the same hole the question, memory and passage bounds had just been closed for, sitting one line above them. It is gone; _request always sends MAX_TOKENS. Unlike the extractor, this client never re-asks a truncation at a raised ceiling, so nothing wanted it.
• PROMPT_VERSION and SCHEMA_VERSION were decoration. They existed for E5's transcript and E6's measurement, and nothing read them — which is exactly T1's failure with a template version: pnk doctor compared a version against the installed one for ten releases while the files that version denoted changed underneath it, and every KB recorded a reference that matched and meant something different. So the prompts and both schemas are now hashed and pinned to their version, and a reworded prompt fails the suite until the digest and the version move with it. E6's numbers are about a specific wording; without this, the record of which wording would be a guess.

The pass also promoted DeepCallFailedError's three kinds to constants, for the reason E1 gives for carrying its escalation value rather than re-deriving it: E4 has to tell a refusal from a truncation in order to label a run, and a caller matching a string literal puts the vocabulary in two places.

And a note on the mutation testing itself. The first attempt at mutating max_tokens back into _request reported "54 passed" and I nearly recorded it as a test gap — the edit had silently matched nothing, because ruff format had collapsed the signature onto one line after the change. A str.replace that matches nothing returns the string unchanged and tells no one, which is the defect conftest._rewrite exists to prevent, reappearing in a throwaway script. Every mutation in this increment now asserts its target matched exactly once before running the suite; the retried mutation failed the right test immediately.

Third review pass — two findings, both about what a reader would look for and not find. The shared-rule tests had landed inside tests/test_deep_client.py, so src/pinakes/paid.py — a module both paid entry points depend on — had no test file bearing its name, and the nearest match, tests/test_paid_path.py, is about something else entirely (which modules may import a client, not what they must then do). Split into tests/test_paid.py, with each file's docstring saying which question it answers, and VERIFICATION gains a section of its own for the rules rather than filing them under the client that happens to use them.

The second: AnthropicTransport.max_retries was a property nothing asserted. build_client_kwargs() == {"max_retries": 0} is tested, and says nothing about this transport passing it — the same gap in a smaller frame as the classifier's. Constructing the client is offline, so the test is not paid-marked; it skips with a reason where [claude] is absent.

Fourth review pass — the sentence this increment falsified was in the file it edited. extract/claude.py's first line read "the only module in src/ permitted to import anthropic", and AnthropicTransport's docstring said "the only place anthropic is constructed". Both were true when written, both became false in the same commit that added the second entry to the allowlist, and neither is reachable by grepping for what changed — the words deep, client and allowlist do not appear in either sentence. Found by grepping for the claim shape ("only module", "the only place") rather than for the diff, which is the same move that turned up docs/ROADMAP.md:139 at E2's handover: a stale pointer has no wrong text to search for, only a claim that is no longer true.

A Ctrl-C voided a call that may have billed — in both paid clients, for as long as either existed

Found by working an exit criterion nobody had tested. E4's plan asks that "interrupting mid-loop leaves a reservation pnk budget reports as unknown outcome, never a lost record". Nothing asserted it, so it was probed with a transport that raises KeyboardInterrupt, and the ledger came back voided — EUR 0 recorded for a request that had already been sent.

Why it was invisible. Every deliberate branch was right. billed_call classifies a timeout as billable-unknown, voids a 429 that never billed, and — in the deep client — catches Exception for anything unclassified and leaves it unresolved. A KeyboardInterrupt is not an Exception, so it fell past all of it into ledger.paid_call's finally, whose job is to close an unfinished call and whose default is to void. Every layer behaved as written. The extractor was worse and had been since I7b: no catch-all at all, so an ordinary defect voided too.

Three things worth keeping:

• The likely interrupt is the one nobody models. A paid run is slow, visible and cancellable; Ctrl-C during one is the normal way it ends when a user changes their mind. It was the only exit path with no test.
• A safe default one layer down is not a safe default. close_unfinished voids because most unclosed calls never billed. That is correct there and wrong here, and the caller is the only place that knows which. except BaseException is what says so.
• The sibling had it too, and fixing one would have been the defect surviving. One invariant, two call sites, two identical clauses — so both moved in the same change, and both tests raise a BaseException rather than a RuntimeError, because a narrower one passes against broken code.

The same GUIDE block quoted a retired sentence twice — so the second time it became a gate

E1 rewrote pnk search's escalation notice and left docs/GUIDE.md displaying the sentence it had just replaced. Its retrospective recorded that nothing in the repo could have caught it. E4 rewrote the same sentence and left the same block stale again, caught only because someone grepped for the old wording before shipping.

Why every existing check is blind to it. The prose is well-formed. Every link resolves. mkdocs build --strict is green. tests/test_verification.py checks that named tests exist, not that quoted output is current. A fenced block showing a previous build's output is correct Markdown describing a program that no longer exists, and nothing in this repo reads it as anything else.

The checkable half is the negative one. Diffing a fenced block against real output would need the command, its models and a corpus. "Every printed constant must appear in the docs" is simply false — most should not. But a sentence this build can no longer print must appear nowhere, and that is a grep. tests/test_docs_quote_the_shipped_sentences.py holds the retirement list, one row per retired sentence with what replaced it; retiring a sentence is a deliberate act, so adding the row is part of it.

Two things that make the gate honest rather than decorative. It searches src/ as well as docs/, because a retired sentence surviving in a docstring is the same defect one layer in — and this project's docstrings are where its reasoning lives. And it was run against the pre-E4 tree, where all four rows fail: a gate never observed failing is a gate nobody has tested.

E5 — The run transcript (20260812 05:11)

MEDIUM — a mutation restored a file and silently took a real edit with it. The mutation pass works by editing a source file, running the suite, then git checkout <file>. Between the first commit and mutation 5 there was exactly one source edit — a comment in cli.py correcting a claim about clear_cache_paid — and git checkout src/pinakes/cli.py reverted it along with the mutation. Nothing failed: the tests pass either way, ./check.sh is green, and the reverted text was a comment, so no gate could see it. It was found by reading the increment's own diff in the adversarial pass and noticing the comment said what the earlier draft said.

The lesson is narrow and mechanical: git checkout restores to the last commit, not to the state before the mutation. Either commit before mutating, or restore with git stash/a copy. This increment did the former for the code and got caught by the one edit that landed in between.

And it happened a second time, two passes later — git checkout tests/free_path_run.py, run to undo a deliberate deletion that was proving a gate row discriminates, reverted the uncommitted addition of that same call. Knowing the trap was not enough to avoid it; the only thing that caught it the second time was checking for it immediately afterwards, because it was now expected. Write the check, not the resolution: after any git checkout <file> in a mutation loop, grep for the thing that was supposed to survive.

MEDIUM — a test that asserted the wrong half of its own claim. "The temp file is .tmp, not .json, so a killed write leaves nothing the readers count" was tested by planting a .tmp-abcdef.tmp file and checking the glob ignored it. That proves the glob ignores .tmp files. It says nothing about what the writer names its temporaries, and it kept passing when the suffix was mutated to .json. It now spies on os.replace, whose source argument is the file a kill one instruction earlier would have left behind, and asserts on that name.

Generalisable: when a test plants the input it then checks, ask which half of the claim it actually reaches. The planted value came from the test's understanding of the code rather than from the code, so the two could disagree without the test noticing — which is the whole failure mode the mutation pass exists to find, and it found it.

LOW — the confirm-then-re-call path had never been tested, on either store. sys.stdin.isatty() is False under pytest, so every --clear-cache test since I4 took the --yes route straight past the prompt, the y/n branch and the second sync() call. Three tests now walk it. Worth recording because the gap was invisible in the ordinary way: the flag had coverage, the interactive flag did not, and no coverage number distinguishes them.

A decision worth writing down: --clear-cache=transcripts names a store, and the two values before it name authorisations. --clear-cache and --clear-cache=paid both clear the whole extraction cache and differ only in what they permit — a documented distinction, with a comment in cli.py explaining why the bare form is not called =free. Layering transcripts onto that axis would have meant --clear-cache=transcripts also emptying the extraction cache, which destroys more than the flag names. Mixing the two axes in one flag is a real cost, and it was paid deliberately rather than by accident: D-26 asked for a --clear-cache target, and a target is what it is.

What E6 inherits. The transcript is the record the measurement run reports out of: it carries call_ids, the estimate and the reconciled spend per run, so the over-reservation factor E6 must publish can be computed from the files a measured run leaves behind rather than from a spreadsheet kept beside it. transcript.call_ids() plus sync.ledger_spend() is the join, already written for the confirmation prompt.

E6 — A seam the tests never crossed (20260812 06:41)

CRITICAL — pnk ask --deep has never worked against the live API, and 0.22.0 through 0.25.0 all ship it. The first real --deep call E6 made returned a 400 before it billed anything:

output_config.format.schema: For 'integer' type, properties maximum, minimum are not supported

deep/client.py: answer_schema emits {"type": "integer", "minimum": 1, "maximum": passages} on every answer call, and structured outputs does not accept numerical constraints. A second probe found subproblems_schema's maxItems is refused the same way, so the decompose call was broken too — fixing the answer call alone would not have produced a working loop.

The lesson is about the seam, not the schema. E3 introduced Transport so the whole loop could be driven from recorded fixtures with anthropic absent, and E4 and E5 were both tested that way, green throughout. That is a good seam and it bought real things. What it also did was guarantee that no test in the suite has ever sent a schema to the API — the one field the API validates and the fixtures cannot. A fixture asserts what we believe a response looks like; it cannot assert that the request was acceptable. Every layer above the seam was correct, which is exactly why nothing went red: the defect lived in the only inch of the path the seam removes.

The generalisation: a seam introduced for testability defines a region the tests cannot reach, and that region needs its own gate. For this one the gate is cheap and fixture-free — assert the two schema builders emit no keyword on structured outputs' documented unsupported list. It would have failed at E4, on a branch that was green.

Three things behaved exactly as designed and are worth recording, because a bad failure is where you find out. The accountant reserved, refused and voided: the failed call billed €0.0000, so a run of 400s cannot consume a budget. The error surfaced the API's own sentence rather than a paraphrase, which is what made the cause obvious in one read. And the blast radius was genuinely bounded to the one path — pnk search, pnk ask without --deep, and the paid extractor were all untouched, because none of them shares this client.

Running the instrument found two bugs in the instrument. tools/deep_reservation.py count filtered its token-count payload to model/system/messages, dropping output_config — so it measured PROMPT_TOKENS with the schema excluded, a fraction of the real figure. And QUESTION_TOKENS differenced an arbitrary 200-word probe, so it returned ~200 by construction: it measured the probe, not the question ceiling. Both would have published a wrong constant with no symptom. A measurement probe is code and earns the same adversarial pass as anything else — two of five constants were measured wrong on the first run, and reading the script is what found neither; running it is.

E6 — The measurement run (20260821 07:53)

HIGH — the instrument that publishes the numbers had no tests, and its --json had never run. vars() on a slots=True dataclass raises TypeError, so tools/deep_reservation.py count --json and report --json both failed on their first row, in every version that ever existed. It survived because every measurement session read the printed table instead. The general shape is worth keeping: a tool written to be run by hand has exactly one exercised path — the one the author happened to type — and the flag nobody typed is not a lesser-tested path, it is an untested one. The fix is cheap; noticing was the whole cost, and what noticed was sitting down to write the tests rather than any run of the tool.

HIGH — the published factor was not reproducible, and nothing said so. The handover recorded 19.0× on synthesis and 16.5× on the loop. Those came from measurement KBs under /tmp, which the operating system reaped after nine days, taking every transcript and every ledger row with them. The money was spent and the numbers were real, but by the time anyone re-ran report the evidence for them was gone — and report cheerfully printed a different factor over the surviving records without any indication that it was answering a smaller question. A measurement whose substrate lives in /tmp has a shelf life, and the number outlived it. What made this recoverable rather than merely lost is that the runbook's rebuild step is free: the KBs come from tests/demo-kb and tests/partner-kb, so the run could simply be done again.

HIGH — the runbook's own step (c) measures the wrong branch. It names three no-answer questions to exercise the loop. On the calibrated KB one of them — "Which software does the catalogue run on?" — scores medium, which takes the cheap branch: running the runbook as written buys a synthesis call and records it as a loop measurement. The free pre-flight the same document prescribes is what caught it, one paragraph after the document tells you to run it. Step (b) already warns that "a 2 means the branch was mis-selected and the run measured the wrong thing"; the inverse case had no such check, and the questions were never re-verified after the thresholds were fitted.

HIGH — and the reason step (c) chose those questions is itself false. It argues that no-answer questions are the right instrument because "nothing in the corpus answers them, so the sufficiency gate cannot stop the run early and it goes to the round cap — which is the worst case the reservation was sized for, and therefore the only case that measures it." Measured: both decomposition runs stopped at sufficiency, after 2 rounds and after 1 round of 3. A sufficiency gate reading a calibrated signal is perfectly willing to conclude that enough has been established about a question the corpus cannot answer. So on a calibrated KB the loop's worst case is not reachable by choosing a hard question, and the branch that actually reaches the round cap is unknown, on a KB with no thresholds at all — the branch the runbook does not mention.

MEDIUM — the more calibrated the KB, the more over-reserved its loop is. The three branches came out 29.75× (synthesis), 50.92× (calibrated loop) and 22.35× (uncalibrated loop). The ordering is not noise and it is not a defect: a reservation must cover max_rounds, and calibration is exactly what lets a run stop before it gets there. The uncalibrated branch is the least over-reserved because it has no early stop and spends the rounds it reserved. Reporting a single blended figure would have hidden this entirely, which is the argument D-28 made before any of it was measured.

MEDIUM — MAX_TOKENS carries most of the over-reservation, and is the constant least safe to lower. 8,000 reserved against a widest-observed 660 across 22 reconciled calls (mean 241) — 12.12×, against 1.50× to 8.93× for the five input constants. It dominates because output bills at five times input and is two thirds of a round's price under the shipped defaults — and four fifths at the narrower geometry these runs actually used. It is also the only one of the six that is a truncation rather than a bill: an input ceiling set too low over-reserves, while an output ceiling set too low cuts an answer off mid-sentence. The temptation to lower the one constant that would visibly improve the headline figure is therefore precisely inverted from where it is safe to do so, which is worth stating plainly next to a 12× ratio.

LOW — the [budget] block the runbook tells you to append is now two-thirds the shipped default. It prescribes per_operation_eur = 2.00 and daily_eur = 6.00; D-30 raised the defaults to exactly those values in 0.24.0. Both measurement KBs ran the whole plan with no [budget] section at all. Harmless, but it is a hand-editing step the document still asks for and no longer needs — and the two keys that do still differ (confirm_above_eur, monthly_eur) are the two it does not explain.

LOW — the refusal path had never been run, and works exactly as specified. Step (d), untouched through two measurement sessions. With per_operation_eur = 1.00 against the loop branch's €1.38 estimate: refused before the first call, exit 1, no ledger row (22 lines before and after) and no transcript (7 files before and after), with a message naming the cap, the headroom, the branch, the call and round count, the complete manifest edit that would admit the run, and the two cheaper routes before raising a cap. D-23 and E5's "a run that never returned writes none" both hold.

A prose tool pointed at a comment run containing load-bearing whitespace (20260821 22:48)

Found by the E6-close adversarial pass, after ./check.sh was green — twice. While annotating estimate.py's constants, textwrap was used to reflow the affected comment runs to fix line-length errors, and it flattened an indented \-continued shell command into ... \ tests/demo-kb/docs on one line — a literal backslash argument for anyone who pastes it. Caught before the tag, fixed in 4d5debf.

The class, and why no gate can see it: the result is legal, correctly-lengthed comment text, so ruff accepts it; it is a comment, so pyright never reads it; and a diff review reads the new prose rather than the old command. No assertion anywhere could have failed — which makes it a clean example of what the mutation battery cannot reach, now noted in docs/BUILDING.md § 4. The rule: a prose tool's output over text containing load-bearing whitespace is re-read as the thing it is — a command, a table, an indent — never as prose.

The sibling case, from the same day's release sweeps: a script that navigates Markdown by a heading prefix steps over every shallower heading. 0.25.3's sweep inserted its ROADMAP section with a startswith("## ") scan for the next section — and # Part 5 · What is not built starts with "# ", so the scan stepped straight over the divider and the Part heading and placed a finished release section inside the unbuilt-work part. release_order_gate could not see it: it checks order among release sections, never which part they sit in. Diagnosed by the sweep's own author from the script, not guessed; the other three insertions from that sweep were each anchored immediately after a located line and are correctly placed — the one that navigated forward to find its slot is the one that crossed a boundary it could not represent. Moved in 0.25.4. The rule both halves of this entry share: a prose-shaped tool applied to a file whose structure carries meaning it does not model needs its output re-read against that structure — heading level, load-bearing whitespace, a divider — never against the text alone.

The same pass also found a shorthand ("output is two thirds of a round's price") that is correct beside the shipped defaults and wrong at the measurement KB's geometry (four fifths there); it was qualified at its five new uses only, deliberately — the pre-existing uses sit beside default-geometry arithmetic, where the shorthand is exact, and a blanket qualifier would cost every reader to protect none.

E7 — Printed suggestions, and the mutant that proved the battery blind (20260822 01:24)

The increment that closes the deep release. A --deep run ends by printing the links[] entries its own citations propose. Small — one module, one CLI seam — and it turned up four defects the tests could not see, three of them found by mutating rather than by reading.

HIGH — the control mutant survived, and it was the first one run. docs/BUILDING.md § 4 says to kill a known-catchable mutant first, because a run with no kills is a broken harness rather than a clean bill. This run had seven kills and lost the control: REL = "co-cited" → "related" left all 71 tests green. Every assertion about the relation imported REL and compared it with itself, so the constant and the expectation moved together. The shipped value of the thing a user pastes was pinned by nothing.

The generalisation is worth more than the fix: a constant imported into both sides of an assertion is not tested, it is restated. test_cli_ask.py's FINGERPRINT comment says exactly this one file away — "a test that derives the expected value from the same object it checks would still pass if fingerprint() started returning the empty string" — and this suite was written without noticing it applied. One literal test now names ("co-cited", "deep"). Half of it was already caught by accident: rel = "" went red, because sidecar._links refuses an entry without one. Only the plausible wrong value survived, which is the direction that matters.

HIGH — a test that certified containment was satisfied by absence. The escaping-path test cited ../outside.md, a path with nothing at the end of it, so the read failed because the file was missing and the assertion passed with no containment check at all. The mutation pass is what separated them: the bypass mutant killed the neighbouring deleted-document test and left this one green, on the same code. It now cites a real document in a second KB next door, with a sidecar carrying the id the citation claims — so only containment can refuse it, and what it prevents is nameable: a fragment carrying pnk://<this KB>/<that KB's document>.

MEDIUM — three direction tests could not observe direction. ULIDs are monotonic, and the fixture built its documents alpha, beta, gamma, so the ids ascended in the same order as the paths. Every assertion about which sidecar an entry lands in was green whether the code ordered by path or by id. Minting the fixture backwards made the two orders disagree — and immediately paid: the entry order inside a sidecar was by URI, which is mint order, which is arbitrary to a reader. A fixture whose two orderings agree cannot tell them apart, and the agreement is usually accidental.

MEDIUM — a newline in a filename would have broken the fragment, silently. The module builds YAML as text, which is defensible: every value in it is a ULID URI or one of two constants, and a test pins that against sidecar.needs_quoting. But the document paths go into YAML comments, and a POSIX filename may contain a newline — one ends the comment and turns the rest of the path into a node. needs_quoting cannot see it, because it answers about a scalar and a comment is not one. The lesson is the sibling of 0.25.3's reflowed shell command: a value that is safe as a scalar is not thereby safe as a comment, and the check that certifies the first says nothing about the second.

Two more, from reading the diff as a stranger. propose is public and re-checks every endpoint it is handed, but would have proposed a document linked to itself — which pnk link refuses outright. And resolution was quadratic in disk reads: a block citing n documents makes n(n-1)/2 pairs, each resolving both endpoints, so every sidecar was read and YAML-parsed n-1 times, with [retrieval] final_k having no ceiling and the money already spent.

The design decision worth keeping: observing and proposing are two functions, and the split exists so a test can exist. § 5's rule is that a suggestion's endpoints must be documents this run retrieved. Enforced inside one function, the rule is unfalsifiable — every candidate comes from the same expression that validates it, so a working guard and a missing one produce identical output on every input a test could construct. co_citations observes; propose re-checks what it is handed; the refusal test calls propose with a pair no run would produce. A guard whose input is built by its own validator is not a guard, and its test is a tautology in test clothing.

What the battery still cannot reach. The prompt-injection test — a retrieved passage instructing the model to add a link, obeyed in prose, producing nothing — asserts the absence of a behaviour. No mutation makes it fail, because the code that would fail it was never written: nothing in the module reads AnswerBlock.text. It is a real test of a real property and it is not mutation-backed, which is the same class docs/BUILDING.md § 4 names as a defect with no assertion anywhere, seen from the other side.

And an old row that had been false for four releases. docs/DESIGN.md §9 still bounded --deep with "no orchestration the free path doesn't have" — written before the loop existed, and contradicted by the loop the moment it shipped. docs/graph/PINAKES_APPROACH.md § 6 had asked for that exact row to be amended in the increment that shipped the design, and named the replacement bound. E4 shipped and the row did not move. Found here by auditing the neighbourhood rather than the diff — the increment that closes a release is the last cheap chance to fix what the release made false, because after it nobody is reading those rows for a while.

Ten of ten mutants killed after the fixes, control included, with the containment mutant now caught by the containment test rather than by its neighbour.

The mutation harness — turning the last unguarded step into a tool (20260822 06:14)

plans/20260821_0745-mutation-harness.md, in one increment. The precedent is tools/land.py: when prose has failed repeatedly against a class of mistake that fails silently, the rule stops being a rule and becomes a tool.

HIGH — a skipped test and a passing test are the same exit code, and this repository skips constantly. Not in the plan; found by measuring pytest before writing any code. A selector whose tests all skip exits 0, which is byte for byte the SURVIVED signal, and Pinakes skips on a missing extra by design (pdf, paid, model, the three-leg CI matrix). A battery aimed at one of those on a [light] checkout would have reported every mutant in it unpinned — a confident, wrong claim about exactly the assertions a mutation pass exists to check. The mirror case is worse and equally invisible: a selector that is already red reports KILLED for every mutant aimed at it. One pre-flight run per selector closes both, before any file is written.

HIGH — SIGTERM skips finally, so a try/finally restore is not a restore. Measured, 1 of 1. SIGINT already raises KeyboardInterrupt, which is why the hazard is invisible to anyone who only tests with Ctrl-C — and why the first version handled only SIGTERM until the harness's own battery showed that dropping SIGHUP and SIGQUIT was a one-word edit no test could see. SIGKILL cannot be handled at all, which is the real reason the plan's refuse if the target differs from HEAD step is not a formality: after a hard kill the only recovery is git checkout <file>, and the refusal is what makes that recovery correct rather than the seventh instance of the trap.

MEDIUM — the T3 trap reproduces 6 times out of 6, and its test must not race the clock. A same-length mutant (min(value, MAX) → max(value, MAX)) written in the same wall-clock second as the previous compile passes every test, because CPython validates a .pyc on (mtime-to-the-second, size). Reproducing it first is what made the test honest: the obvious test — assert the same-length mutant is KILLED end to end — goes green on a slow machine with the invalidation deleted, because the second boundary is crossed anyway. Two tests replace it. One asserts no bytecode cache exists during a mutation, which has no clock in it at all; the other forges the stale condition with os.utime and watches the mutant vanish, so the clearing has a control rather than being ceremony.

MEDIUM — pytest's <error> tag covers two opposite events, and conflating them threw away real kills. A collection error is the invalid mutant: nothing ran, no assertion was tested. A setup or teardown error is a real node the mutant broke on the way in or out — and in this repository fixtures build indexes, manifests and KBs out of src/, so fixture-mediated detection is the common shape rather than a corner. Treating both as ERRORED, and testing that before failures, reported a mutant that tripped a fixture and failed a plain assertion as "the mutant did not run", tallied 0 killed. The two are now told apart structurally — a collection failure carries no line attribute — rather than by matching pytest's message text.

The correction that mattered more than the finding. The reviewer who filed that one also proposed the fix: route setup and teardown errors into KILLED. An independent skeptic, asked only to refute the finding, confirmed the defect and rejected the remedy — routing them into KILLED would manufacture the false green the tool exists to prevent, because no assertion fired and nobody may write "pinned by test X" for a fixture noticing something. The shipped fix keeps the conservative direction and repairs only the sentence. An adversarial pass is worth more when the verifier is allowed to disagree with the finder's remedy as well as with the finding.

The shape that recurred three times: a guard the CLI cannot reach. Pointing the tool at itself found, in three separate rounds, four clauses that no battery-driven test could kill — the second anchor check inside applied(), classify's timed_out and setup_errors branches, and the report's only-a-KILLED-row-may-name-a-killer guard. Each is redundant against the code as it stands today and each guards a state one line of drift away. Every one surfaced as a SURVIVED row, which is the row the tool exists to print; none would have been visible to a reviewer reading the diff. They are kept, and each gained a direct test on the constructed state rather than being deleted as dead code — redundancy nothing tests is indistinguishable from redundancy that has quietly stopped working.

Twice, the battery's own selectors were the thing that was stale. Both times a guard read SURVIVED because the battery still named the test that existed before the fix, not the one written for it. The lesson is narrow and mechanical, and it is about the battery rather than the code: a battery is source that goes stale like any other, and a SURVIVED row is a claim about a pair — the mutant and the selector — either half of which can be wrong. The third time, the anchor pre-flight caught it before a single mutation ran, which is the whole argument for hoisting that check.

What the tool still cannot see. A survivor is a claim that wants checking by hand — the record holds two increments where a mutant that did not reproduce the real prior logic was briefly taken for a result, and the E7 session's own control mutant survived because every assertion imported the constant it compared against. A mutation result implicating a file the mutant never touched is a result about the harness, not the code, and nothing here detects that.

The exit criterion, met the only way it could be. The tool was run against itself: 25 mutants, each disarming one of its own guards, 25 killed, each by the test named beside it. Before that, two mutants of src/pinakes/graph/traverse.py's caps, both killed, tree clean afterwards — because a tool that has only ever been run on its own fixtures has not been run.

A gate cited by a procedure it cannot read (20260822 06:35)

docs/RELEASING.md's sweep table names five places a release stales. One is STATUS's Published on PyPI prose. Its "where the new entry goes" row answers: python3 tools/release_order_gate.py decides it. No pattern in that gate matched the list. So the procedure delegated a placement decision to a check that could not read the document, and the green line 5 sequences in release order was read as covering a list it had never opened.

It drifted, as delegated-to-nobody things do: 0.25.1 → 0.25.3 → 0.25.2 → 0.25.4, wrong on SemVer and on verification time, surviving every green run from 20260821 to 20260822.

The generalisable shape is not "a missing pattern". It is a citation nobody checked. A document naming a tool as the authority for something is a claim about that tool's coverage, and it is exactly the kind of claim that is written once and never re-read. Grep the other direction occasionally: for each gate, what do the documents say it covers, and does it?

Three things that fell out of building it, each worth more than the fix:

• A gate reads an order; it cannot see a count. Landed the same day, from the other side: a concurrent session appended "thirty-seven" to a sentence that still said "thirty-six", one line apart, through a green check.sh, a green make docs and a green release_order_gate. Caught by grepping the neighbourhood, not by reading the diff — the wrong half was context, so it never appeared in the diff at all. Counts stay a documentation rule for that reason.
• "Tolerate it" needs a direction, or it is a hole. This list is legitimately short between a release landing and its verification, because a claim about the index is held back until it is verified from the index. Exempting it from agreement is right; exempting it from direction is not. It may lag every other sequence and may never lead one — a paragraph about a release the CHANGELOG has never heard of is a claim nothing else records. The first draft had the exemption and no direction, and nothing could have told the two apart.
• Say what the exemption costs, in the gate. Because a missing newest entry is legal, an entry written in a shape the pattern does not match is indistinguishable from one not yet written — silently unchecked. The floor catches wholesale rot; it does not catch one mis-shaped newest entry. Undocumented limits get trusted past.

And one environmental trap, found before it could produce a false green: this repo has no .python-version, so uv gave a fresh worktree CPython 3.14.7 while the primary checkout and CI run 3.13. A green ./check.sh in a new worktree is therefore not evidence about the interpreter anything else uses. Pinned by hand here; the root fix is a proposal, not this increment's.

Verify the remedy, not only the finding (20260822 07:25)

docs/BUILDING.md § 5 already says a fix applied under review inherits the review's confidence and none of its scrutiny. This is its sibling, and it is about the review pass rather than the commit after it: a correct finding arrives with a proposed remedy attached, and the remedy is the half nobody re-checks. The finding survives scrutiny; the remedy rides in behind it.

Two instances the same night, in two sessions working concurrently, each caught only because the second pass argued with the fix rather than stopping at the defect:

• A remedy that would have manufactured a false green. A classify bug was real and confirmed. Its proposed fix would have made the check report success on input it had never examined — a repair that removed the symptom by removing the observation.
• A remedy that was right and unbounded. STATUS's Published on PyPI list is legitimately short between a release landing and its verification, because a claim about the index is held back until it is verified from the index. Exempting that sequence from the release-order gate's agreement check was correct. Exempting it without a direction was a hole with a docstring: no input could distinguish a working exemption from a missing one. It became may lag, never lead — the list may fall behind every other sequence and may never name a release the CHANGELOG has not heard of.

A review culture that re-checks only findings ships both of these at full confidence. Both were defensible, both were proposed by the agent that had just been right about something, and that is exactly what makes a remedy hard to doubt.

The cheap test: state what input would look different if the remedy were absent. If none exists, the remedy is unfalsifiable and the test written for it will be a tautology — the same failure E7 recorded, where a guard whose input is built by its own validator cannot be shown to guard anything.

The cheap structural version, if only one thing survives from this: when an adversarial reviewer is asked to refute a finding, ask it to refute the proposed fix as a separate verdict. Both instances above were caught that way and neither was caught any other way — one by accident, when a skeptic exceeded its brief, and one by design, when the second pass asked tolerant in which direction?. It costs one more question per finding. The mechanism and the first instance came from the concurrent session; this fragment shipped without them and had to be amended, which is the argument in miniature.

And it worked because reviewing was somebody's assigned job. That amendment was found by a session whose remaining task was to look at work other people had finished — development split to one agent, planning and review to another. A review that happens as a courtesy after the work is done is the one that gets skipped when the work runs long; the catch here depended on it being nobody's optional extra.

A coda from the concurrent session, worth keeping beside this. The night's most serious defect — pnk serve raising ModuleNotFoundError on every fresh install of every published version, from a lower-bound-only dependency pin that 37 --frozen CI invocations never resolve — was not found by a test, a gate or a review. It was found by someone asking what should we build next and installing the product to answer it. docs/BUILDING.md warns that a test seam defines a region no test reaches; here the region no test reached was the product itself. Reproduced independently in both sessions before it was written down.

Capping mcp, and the leg that had to be run against the defect (20260822 08:20)

HIGH — a CI leg written to catch a defect is a claim until it is run against the defect, and the mutation battery cannot reach this one. No mutant of any tracked file makes uv resolve mcp 2.0.0: the thing under test is a dependency resolve, and tools/mutate.py mutates bytes in the tree. So the leg was verified the only way left — git archive HEAD | tar -x into /tmp, the cap deleted from the copy, uv build, and both new steps pointed at the resulting wheel:

wheel-import: 1 module(s) did not import against the resolved dependency set:
  pinakes.serve: ModuleNotFoundError: No module named 'mcp.server.fastmcp'

and pnk serve exited 1 with no "serverInfo" anywhere in its output. (That block is the run's real stdout, pasted. The first draft of this fragment paraphrased it into two module lines — in the fragment whose whole thesis is that the artifact has to be run. Caught by review; a quoted output that was retyped is a claim wearing evidence's clothes.)

Without that, "the leg would have caught it" is a statement about a check that has only ever been run against a fixed tree — the shape docs/BUILDING.md already refuses for a test ("pinned by test X is a claim about a failing test"). The generalisation: when the battery cannot reach a guard, build the broken artifact by hand and point the guard at it. It cost about four minutes.

HIGH — the review's own remedies were where the remaining defects were, exactly as verify the remedy, not only the finding predicted the night it landed. Two of the three worst findings in this increment were not in the first draft's logic but in the mechanisms written to guard it:

• An allowance keyed on the library forgave the module the gate exists for. --allow-missing pypdfium2 excused any module failing on pypdfium2 — including pinakes.serve, had anything on its import chain ever reached it. Demonstrated against a synthetic package: serve did not import and the gate printed a clean pass. An allowance now names the module and the library, and a --required module may never appear in one.
• The "the gate can still fail" step was satisfied by an environment where the gate never ran. It grepped the tool's generic failure headline — which the tool also printed when the package itself was not importable, i.e. when --with installed nothing (a mistyped extras syntax does exactly that; measured). The two branches now print different headlines and the step names the one failure the bare wheel genuinely has. The defect was inside the step written to prevent that class, and a test pinned the collision rather than catching it.

The fragment's cheap test applied to this increment's own thesis — what input would look different if the remedy were absent? — is the uncapped wheel above. It exists, and it is the answer to the question this increment turns on: without it, the cap is the only real change and the CI leg is decoration.

HIGH — and then the remedies for those had defects, in the same shape, found by a second review that was asked only to refute them. Three rounds, each finding less than the last, and the third still finding four things that mattered:

• The guard against a missing wheel failed open. set -- dist/*.whl; test -f "$1" && test $# -eq 1 — under set -e, a failing first command of an && list does not abort, so an empty dist/ fell through to wheel="$PWD/dist/*.whl" and carried on. The two-wheel case aborted, which is why it looked right. It is an if now. Two sessions found this independently within minutes of each other; it came in as a review suggestion and was implemented verbatim.
• make smoke exited 0 while printing a traceback. pnk serve … | grep -q '"serverInfo"' — grep matches, closes the pipe, the server dies on BrokenPipeError, and the pipeline's status is grep's. The target printed "answers an MCP handshake" over a crashed server, and a changelog entry claimed it ran the same checks as CI. Output to a file, then grep it twice.
• continue-on-error: true defeats an assertion written against if:. The test said a gate that can be switched off is not a gate and then checked one of the two ways to switch one off. On the step in front of uv publish, where being wrong is a version number PyPI will not take back.
• The source-tree refusal was scoped to the checkout the script lives in. This project mandates a worktree per change, so another checkout's editable install is a path the gate has never heard of — demonstrated with one checkout's interpreter and another's gate: a clean green pass over src/pinakes. A negative check has to enumerate every wrong answer; the positive one — the package must be inside a site-packages or dist-packages directory — has one right answer, and that is the difference between the two shapes.

And a declaration test only pins the file the last edit touched. --min-modules 50 sat at four call sites across three files; deleting it from any one of them survived every test, and the tool then falls back to a floor of 20 against a package of 57 modules. The assertion now iterates the invocations rather than grepping the tree, which is the same lesson as read the sequence, not the neighbourhood in docs/RELEASING.md: a property of a set is not a property of any member of it.

MEDIUM — a gate that reads a range of lines reads the prose inside the range. check.sh's extras-not-core gate is awk '/^dependencies = \[/,/^\]/' pyproject.toml | grep -qiE 'pypdfium2|anthropic'. The comment added above mcp — explaining that anthropic was measured and deliberately not capped — turned it red and reported anthropic as a core dependency. The gate had been correct for a year only because nobody had written a comment in that block. Same class as 0.25.3's textwrap reflow of a \-continued shell command: a line-shaped tool applied to structure it does not model. The parsed-side test never had the problem, because it reads the requirement list rather than the lines.

MEDIUM — nothing resolves a test path written in a comment. check.sh and tests/test_paid_path.py both pointed readers at test_packaging.py::test_paid_and_pdf_clients_stay_out_of_core. That test has never existed under either file's history; the real name is test_extractors_stay_extras. docs/VERIFICATION.md is gated for precisely this reason — a table of test paths is prose until something executes it — and a comment is prose that nothing gates at all. Both fixed; the class is open everywhere else.

MEDIUM — measure the sibling before capping it, and measure the surface you actually call. Capping all three lower-bound-only requirements was the obvious response and would have been wrong. Both siblings were measured: anthropic 1.0.0 and sentence-transformers 6.0.0 keep every constructor parameter, response-model field (Message, Usage, MessageTokensCount) and method signature (encode, predict, get_sentence_embedding_dimension, max_seq_length) that src/ touches. The first pass measured only the constructors, and review caught that. The gap mattered: extract/claude.py consumes response.model_dump() as a dict, so a renamed Usage field would compute a cost of 0 — the spending guard silently disabled, with no exception anywhere. A constructor signature says nothing about that. The remedy for the class is testing the resolve, not capping on reflex; a project that caps everywhere buys the same silence with a different cause.

The battery is what turned four rounds of prose into four rounds of evidence, and the survivor count went the wrong way twice before it went the right way.

Round	Mutants	Survived	Where
1 — the increment	11	0	—
2 — after review 1's fixes	21	3	tests written from the fix's own description
3 — after review 2's fixes	26 (reviewer's) → 33 (mine)	12 → then 2	the remedies' own surface
4 — after review 3's fixes	25 (reviewer's) → 40 (mine)	19 → then 0	the remedies' remedies

Round one said 11/11 and meant it about eleven assertions, which is all a battery ever means. Every later round found survivors in the surface the previous round's fixes had added — exit 1 changed to exit 0 in a step that then printed ::error:: and reported success; a continue-on-error on the job under tests that guarded the steps; --min-modules 1 under a test that asserted the flag was present; a leading - in a Makefile recipe, which is make's own way of ignoring an exit status. A remedy is new code, and new code nothing has tried to break is a claim. The count that matters is not 40/40, it is that three consecutive reviews each found something and the fourth round's mutants were written by someone who had not written the fixes.

What the new legs still cannot see, named so nobody reads them as more. Import-time breaks only, on the install states CI runs. A dependency that keeps its module layout and changes a signature passes every step added here and fails on a user's machine. [st] — the default backend — is resolved fresh by nothing, because it is ~2GB of torch. And ci.yml runs on push and pull_request, so a major published on a quiet day is caught at the next push or at the tag, never before: release.yml is where that matters, and it now carries both checks in front of uv publish.

A gate must not derive a constant from the document it polices (20260822 17:52)

Membership's first draft derived each sequence's start version from the sequence itself — "the oldest release it contains" — and justified it in the commit message: the starts are observable and monotonic; they never move backwards, because releases are never deleted. Every clause of that is true and the conclusion is still wrong. The gate does not observe releases. It observes the document. Delete STATUS's 0.2.0 row and the derived start becomes 0.2.1: the sequence is still sorted, still contiguous, still internally consistent, and the gate reports green on precisely the deletion it was built to catch.

It is the same failure the gate's own docstring already refuses one paragraph earlier — the direction is declared per sequence, never inferred, because a badly scrambled file would otherwise elect its own answer. I read that sentence while editing the file and then wrote the inferred version of a different constant. A rule stated about one field does not defend the next field somebody adds. Four declared constants cost four lines and cannot be argued into being wrong.

The general shape, worth checking on any gate here: for each constant the gate uses, ask where it came from. If the answer is "the thing being checked", it is not a constant, it is an echo.

A status read is not a status gated on

CLAUDE.md names this trap in its piped costume: check | tail && git commit reports tail's status, so a failing checker looks green. I hit the unpiped one and it is worth naming separately, because the written rule did not cover it and I had read that rule the same day:

./check.sh > log 2>&1; echo "exit=$?"      # status printed, and read, and correct
git add -A && git commit ...              # runs regardless, on the next line

The status was captured, printed, and true. Nothing consumed it. A commit landed over a red tree — a lint error, caught later — and the log shows a green-looking sequence of steps. The pipe was never the mechanism; the mechanism is that the exit status must be what the next command reads, and a human reading it off the screen is not the next command. The fix is one wrapper:

./check.sh > log 2>&1; RC=$?
if [ $RC -ne 0 ]; then …report…; else …commit…; fi

And a third instance of verify the remedy, not only the finding

The exception mechanism for 0.11.0 was agreed with the planner as a loose match — find the version anywhere in a ## heading. Implementing it meant first finding where that heading actually is: docs/ROADMAP.md:1721, inside # Part 5, with the release table linking to it on purpose. 0.11.0 has no Part 4 section by design. The loose match would have taken that heading as its release section and failed twice on a correct document — placement, because Part 5 declares no range, and ordering, because it follows 0.27.2.

The finding was right: membership needs an exception mechanism. The remedy attached to it was wrong, and the risk that killed it had been named in the same message that proposed it — "matching a version anywhere widens what counts as a release section" — and read by both of us as hypothetical. It was not hypothetical; the widened set already had a legitimate member. A risk stated beside a remedy is not a risk that has been checked.

The mcp 2.x port — the port was four lines, the gate was not (20260822 18:23)

HIGH — A gate whose passing depends on a dependency's teardown timing is not a gate. The handshake in both workflows wrote three JSON-RPC lines into pnk serve and closed stdin. Measured on the same three lines, same KB, ten runs each:

	initialize answered	tools/list answered
mcp 1.28.1	10/10	10/10
mcp 2.0.0	10/10	2/10
a real ClientSession, 2.0.0	8/8	8/8

1.x drained the queue before shutting down on EOF; 2.x does not. Nothing in the step named that dependency, and nothing could have: it is a property of a library's shutdown path, not of its API. The generalisation is worth more than the instance — if a check's passing depends on a race nobody in this repository wrote, it is not a check — and the remedy is not a longer sleep but removing the race's cause, which here meant driving the session with a client that holds the connection open until it has its answers.

The timing is the uncomfortable part. That handshake was written the same morning, in 0.27.2, to catch the mcp outage — and it was written against the behaviour of the version it was about to lose. A gate built for a dependency major was itself a hostage to that major. git log -S '"method":"initialize"' returns exactly two commits: the one that added it and the one that removed it, four hours apart.

HIGH — The third copy is the one nobody looks at. Two workflows were fixed; make smoke — the pre-tag check a maintainer actually runs — kept the dead shape and was red on every run against a healthy wheel. Review found it, and found the second half too: test_make_smoke_exercises_the_wheel_rather_than_only_its_version asserted '"method":"initialize"' in body and passed, certifying "drives a handshake" against a recipe that no longer produced one. When a shape is replaced because it is wrong, grep the whole tree for that shape before believing it is gone — and check what the tests pinning the old copies now assert, because a test written against an implementation outlives the implementation.

HIGH — There is no independent source of a project's own version inside its own checkout. The version test compared what the server advertised against importlib.metadata.version("pinakes"), reaching for independence from pinakes.__version__. [tool.hatch.version] reads that same file, so the metadata is the same constant copied into .dist-info at install time — and uv run does not refresh it when the constant changes. Bumping __version__ to cut the release diverged them at once and reddened two tests, with a message blaming the serverInfo defect the increment had just fixed. A landmine the release commit would have stepped on twenty minutes later.

The lesson is not "use the other source". It is that a longer route to the same origin is not a second opinion, and saying so in the docstring is worth more than the appearance of rigour. The genuine cross-check exists, but only in CI: the expected value comes from the built wheel's filename and the advertised one from a separately installed copy of it.

MEDIUM — An error message is part of the assertion surface. assert EXPECTED in result.stderr looked like it checked that a failure names what was expected. It was satisfied by the gate's own sentence, "Until 0.27.2 this field carried the mcp library's version" — because EXPECTED was 0.27.2. The assertion held with the fix deleted. A version literal in a diagnostic is a value a test may accidentally be reading; the message now names no version.

MEDIUM — The negative form of a rule is not the rule. Two exit-status pins asserted the absence of one literal string, out=$(timeout — the shape the step used to have. | tee, || true, if …; then and a backticked capture all stayed green, and | tee is the likely edit, since the shape it replaced existed precisely to print the gate's output. The positive form — this invocation is a simple command under timeout — is one assertion instead of an ever-growing list of forbidden ones. Generally: when a rule says "the status must reach set -e", assert the shape that makes it true, never the shapes that made it false last time.

MEDIUM — step["run"] is a YAML literal block, comments included. release.yml's pins were bare substring searches over it, so commenting out the commands left every assertion satisfied by the prose above them. Its ci.yml twin had required command lines since 0.27.2 for exactly this reason and the release side never got it — a fix applied to one of two twins is half a fix, and the release path is the half where being wrong is irreversible.

MEDIUM — The only untested branch was the one the gate exists for. A server dead on import — the 0.27.2 outage itself — reached the gate's generic session-failure path, and no test drove it. Also untested: the timeout ceiling and serverInfo.name. All three now have one, and --timeout became a real flag so the timeout test costs a second rather than thirty. Ask which branch the tool was built for, and check that one has a test before counting the others.

MEDIUM — A test can pin a message and leave the behaviour free. The timeout test asserted "no complete session in 1s" and nothing about the clock. The mutation battery wired timeout= to a constant while the message still read the flag, and reported ERRORED rather than a kill — the harness refusing to call a run that never finished either a kill or a survivor, which is exactly what tools/mutate.py was built to do. The test now asserts elapsed time as well, with a loose bound: what must be true is seconds rather than the job.

LOW, and a pattern rather than an item — four false statements, all in prose written to explain why the code was careful. SESSION_TIMEOUT_SECONDS was commented "Ten" beside a value of 30; a docstring said comparing __version__ with itself "would hold with the version= argument deleted", which is simply untrue; tools/wheel_import_gate.py still said uv.lock pins 1.28.1; and a changelog fragment called the flaky handshake "a coin flip that landed the same way for a year" when git log -S puts it at four hours old in a repository 28 days old. Explanatory prose is where false claims accumulate, because it is written once, at the moment of most confidence, and never re-read against the code it explains.

LOW — Two run-the-real-thing checks that no unit test could replace. The three CI legs were run verbatim against a built wheel resolved fresh onto mcp 2.0.0, which is the only way to learn that uv run --isolated --no-project --with <wheel> puts pnk on PATH for a child process the gate spawns. And the gate's timeout was driven with a shim that spawns and stays silent: it exits 1 after 32s and leaves no orphan behind it.

Process — the primary checkout's tree can change under a running gate. A peer session ran tools/land.py while ./check.sh was running in that same directory, and three tests failed with "matched 0 # Part heading(s)" — loud, plausible, and pointed at the file the peer had just touched. Nothing was wrong; a re-run at the new sha was green. Run gates in your own worktree. The primary checkout is where landings happen and its tree can change under a running pytest at any moment; a red run there may be about a tree that no longer exists.

Review economics, recorded because the numbers are the argument. Round one: four lenses over the diff, each finding verified by an adversarial refuter. 31 findings raised, 19 confirmed and 12 refuted — and the refutations were substantive, one of them disproving a claimed process-group escape by reading multiprocessing.util.spawnv_passfds on the machine. Two of the confirmed findings were defects I had already found myself; the other seventeen I had not. The mutation battery grew from 15 to 24 mutants over the same increment, all killed. The refutation stage earned its cost twice over: it is what makes 19 findings worth reading rather than 31 worth arguing with.

When a constant cannot be declared, constrain it (20260823)

The echo class — a gate reading a constant out of the document it polices — turned up four times in one gate, and the interesting part is that the same remedy did not work every time.

Echo	Exploit	Remedy
sequence start = its own oldest entry	delete the oldest row; the start moves and hides it	declare it
lagging ceiling = its own newest entry	delete the newest entry; the ceiling drops with it	bound how far it may lag
Part range = its own heading	append a range to # Part 5 and a misfiled section is legal	constrain it
Part count floor = one below the truth	demote # Part 5; the floor passes exactly	raise it to the real count

The third could not be declared, and that is the lesson. Reading Part ranges out of the headings is why the mapping cannot drift from the document — replacing them with a table in the tool would reintroduce the drift the gate exists to catch. So the echo had to stay and be made unexploitable instead: two Parts may not claim the same version, and the Parts must ascend. # Part 4 declaring `0.8.0` onward is then precisely what stops # Part 5 declaring it.

Declare, bound, constrain — in that order of preference. Declaring is strongest and cheapest when the value is genuinely external. Bounding admits the echo and limits its travel. Constraining leaves the echo and removes the freedom that made it exploitable. Reaching for the first when only the third is available produces a table nobody updates.

A floor one below the truth is a floor with a bypass. PARTS_MINIMUM was 4 against 5 real Parts, so demoting the last heading passed it exactly while handing every section beneath to the open-ended Part above. Floors here are written as "this only ever holds, because things are never removed" — which is an argument for setting them at the count, not below it. A floor with slack is a floor someone can stand in.

And a test that asserts a sentence asserts only that something went wrong. Three instances in one increment, each found by mutation and none by reading:

• a range-form test asserting a failure — satisfied by breaking the form entirely
• placement fixtures guarded by assert "reads ascending" not in stderr — satisfied by a reworded message, and by a second failure appearing beside the one under test
• an ordering test asserting "must ascend with the document" — satisfied with the comparison reversed, because a different, correctly-ordered pair then fires and prints the same words

The positive form in each case: assert exactly one failure and which one — the pair, the Parts, the versions. failures_of() exists for that.

On the audit itself. These were found by four independent lenses over the landed gate, each finding then handed to a separate agent told to refute it — not by re-reading the diff. Two of the four lenses found nothing. The two exploits came from the lens that was asked one question only: for every constant this gate uses, where does its value come from?

A correction is a claim like any other (20260823 02:05)

Two sessions spent 20260822–23 finding one defect shape from four directions, and the fourth instance landed on the correction rather than on the code.

A handoff file said the release-order gate's placement and membership halves shipped in 0.27.1 at ba4d7ae. That was wrong. The reviewing session corrected it to "placement and membership landed after 0.28.0 and are unreleased" — which was also wrong. git describe --contains settled it in one command:

09f5449 (placement)   -> v0.28.0~2
0a8bd38 (membership)  -> v0.28.0~1
ba4d7ae (lag + Parts) -> in no tag

The reviewer's own diagnosis is the thing worth keeping: one true fact was established — ba4d7ae is tonight's merge, not 0.27.1 — and extended to a second that did not follow. The true half made the false half feel checked. The evidence to settle it was one command away and was run only after pushback.

This is the same shape as a fix applied under review inherits the review's confidence and none of its scrutiny (BUILDING.md § 5) and its sibling verify the remedy, not only the finding — one level further out. The correction is the least-reviewed statement in any review, because it arrives attached to a finding that has just been demonstrated, and demonstrating the finding feels like demonstrating the fix.

And a second rule fell out of the same exchange, stronger than the one it replaced. The file's ownership table had a held last by column naming sessions. The reviewer said "put the right name in". The right name could not be kept: the writing session was renamed mid-file, and the two sessions' ListAgents views disagreed about which of them existed. So the column was deleted, not corrected — ownership is by role, with ListAgents for who is live.

When a field cannot be kept true, remove the field rather than maintaining it.

That is the end of the ladder this repository has been climbing all week: declare a constant rather than deriving it; bound it when it must be derived; constrain it when it must be read from the thing it polices; and delete it when none of those can hold it honest. A field nobody can keep true is a lie with a maintenance schedule.

A document's version stamp is the one claim in it nothing can check

docs/GUIDE.md opened with "Every command here was run against 0.2.0 (20260728 16:40)" and was published, unchanged, for twenty-six releases. Nothing was broken by it and no gate could see it: ./check.sh proves the release sequences agree across five documents, mkdocs --strict proves every link resolves, and tools/template_drift_gate.py proves the template matches its archive — none of them reads a sentence claiming a command was run.

The stamp is not a small lie. It is the sentence a reader uses to decide whether to trust the other eight hundred lines, and it was wrong in the direction that costs most: it claimed more verification than had happened. Nine output blocks had drifted behind it, and the two most misleading were the ones a reader would act on — a euro estimate quoted 30% high, and three separate statements that only one surface in Pinakes can spend money when pnk ask --deep had been the second for four weeks.

The pattern is that a stale doc rots fastest exactly where it was most specific. Prose survives a release; a quoted output block does not, because it pins a number, a version string and a wording all at once, and any one of the three moving falsifies it. Nine blocks drifted while the paragraphs around them stayed true. That is an argument for keeping worked examples — they are what catch the drift — and against ever writing one without a way to re-run it.

Two things this pass could not do, and said so rather than faking. The paid --deep transcript would cost real money to reproduce, so it is kept and labelled as a 0.24.0 run with the one figure that has since moved named explicitly. And re-running the two-KB link walkthrough would change every ULID in the section without making a sentence truer. The distinction that matters is between an output nobody re-ran and an output nobody re-ran and did not say so. Only the second is a defect.

A near miss worth recording. The first draft of the label said the paid block's command "is now quoted at €0.20" — the figure measured at -k 2. The block's own command passes no -k, and at the default passage count it quotes €0.21. One cent, and the wrong kind of wrong: two invocations collapsed into one number, which is precisely the invented precision the label existed to prevent. Caught by re-reading the command the block actually shows instead of the one measured beside it.

A gate can be green because it is reading the list next door

The Published versions row drifted four releases while a check named after it ran green on every commit. The check was real, the row was wrong, and both statements were true at once: the gate's sixth sequence reads the Published on PyPI prose, and the row is a different list forty lines down in the same file, under the same heading.

What made it invisible was the naming, not the code. A reader auditing "is the PyPI list checked?" finds a sequence called the Published on PyPI prose, sees it green, and stops. Nothing in that report says which of the two lists under that heading it read. The lesson generalises past this file: when one document holds two lists of the same thing, a gate covering one of them reads as covering both, and the coverage claim is what needs the test — not the ordering.

The corollary for within. Scoping a pattern to a region introduces a failure the line-anchored sequences never had: the anchor can match somewhere else. Zero matches is safe, because an empty sequence trips the floor and the floor's message fits. Two matches is not, because reading the first is a silent choice — the anchor deciding for itself which region it meant, which is the "derived, never declared" mistake this module refuses everywhere else, one layer down. So it is a refusal, not a heuristic.

And a fixture proves scoping only if it would break without it. The tree these tests build writes version numbers after the enumeration and in descending order, mirroring the real cell. A within that stopped scoping reads them into the sequence and the tests go red. Without those trailing numbers every test here would pass over a bare unscoped pattern — the fixture would be asserting the thing it was built to assume.

On the floor: an exact count is declared, and still wrong here. Setting the floor to the 41 versions present would be a literal, not a derivation — but check() skips a sequence whose floor failed, and check_membership then skips it too, so deleting one release would report "the pattern has stopped matching what it names" and never say which release went missing. The precise diagnostic is worth more than the redundant one. Declared-not-derived is about where a number comes from; it does not settle what the number should be.

A constant bounds the damage; a relation catches the start

The first fix made the row a sequence with permission to lag, bounded by MAX_VERIFICATION_LAG. That is a real check and it would have caught both drifts — eventually. Counting the history says how late: 29 commits sat in the window where the row was behind and the bound was silent, and both drifts passed through it on the way to being caught.

The bound is a constant, and the thing it approximates is a relation. The row may lag the release documents because an entry waits for index verification; it may not lag the other list recording that same verification, because there is no interval during which one is true and the other is not. Written that way the rule needs no tolerance at all, and the measurement bears it out: zero false positives in 67 commits, red at the first commit of the drift rather than the eleventh.

The general form: when a bound needs a tolerance, ask what the tolerance is standing in for. A number chosen to be loose enough for the legitimate case is by construction also loose enough for some illegitimate ones. If the legitimate case can be named — here, "both lists are waiting on the same verification" — the relation that names it is both tighter and simpler than the number.

Keep the constant anyway. It is not redundant: it catches a drift in which both lists are forgotten together, which the relation cannot see. Two checks over one sequence, catching different failures at different moments, is the correct outcome — and the test that exercises the deep drift now asserts both fire, so neither can be deleted while it stays green.

Committing the mutation batteries — a design statement nobody had measured (20260823 07:34)

HIGH — the sentence that decided this was an assumption wearing a docstring. tools/mutate.py shipped in 0.27.0 saying "a battery is a per-increment working file, not a portable artifact", in a paragraph whose actual subject was where the pytest key lives. Nothing had measured it, and it then decided the question for two months by being read rather than by being right. plans/20260821_0745-mutation-harness.md enumerates what it deliberately left open and persistence is not among them — checked, not assumed. So this fills a gap rather than overturning a position.

The measurement took twenty minutes. Of 81 mutants across six increments, 78 anchors still resolved exactly once a day to a week later, and every one that broke was one whose own code had changed. The failure mode of keeping a battery is a refusal, not a wrong answer — mutate.py resolves every anchor before it writes anything and refuses on 0 matches and on 2. That asymmetry is what made the question decidable. An argument about whether a claim outlives its proof would not have resolved; what does being wrong cost? did.

HIGH — and the measurement was aimed at the half that had never broken. docs/RETROSPECTIVES.md already recorded which half rots, twice in one increment: "a battery is source that goes stale like any other, and a SURVIVED row is a claim about a pair — the mutant and the selector — either half of which can be wrong." Both my 81-mutant recovery and a peer's five refactor scenarios stress-tested anchors. The selector is what historically went stale. The gate resolves both, and the half that matters is the one the headline number says nothing about. Check the record for which half failed before designing a measurement of the other.

HIGH — the repair loop produced two broken mutants under best-case conditions, and that is the strongest evidence against the model, not for it. Two anchors were repaired here: same day, by the author, with full context. Both widened old from one line to two and left new at the one line it used to replace:

Repaired mutant	What the edit actually did
the prose list loses its exemption	deleted minimum=15, and flipped the flag — two changes under a name promising one, and the floor it fell back to sits on the boundary of that sequence's length, so the test may have gone red for the floor
the prose sequence is read in the wrong direction	deleted starts_at and duplicated minimum=15, — keyword argument repeated, a SyntaxError at import

The second read KILLED, in a batch reporting 0 errored. classify treats a collection error as the invalid mutant, and this module is imported inside the test rather than at module scope, so the SyntaxError arrived as an ordinary assertion failure. A confident row about a question nobody asked — the one thing the tool exists to refuse — produced by the tool, in its own corpus, on the first day. n=1 in the direction an adversary predicted, and it is the honest cost of the model.

refuse_a_mutant_that_cannot_compile now catches the syntax half, before the first write, and it earned itself within the hour: restructuring check_anchors staled two mutants in this increment's own battery and it caught both. ast.parse would not have — it accepts f(a=1, a=2) and compile() does not, which is exactly the case that got through, so a test pins the discrimination rather than leaving the choice of compile looking like a detail. The other half has no guard: an edit that still compiles and changes two things under a name promising one is invisible to everything here. Read the diff a mutant produces, never its anchor.

MEDIUM — a corpus with no inventory can shrink to nothing with every gate green. None of the first draft's checks was a count, and the sanctioned repair for a property that has genuinely gone is to delete its mutant — which makes deletion the cheapest path to green during a refactor. Split a 739-line gate six weeks from now, watch thirteen anchors resolve zero times, delete thirteen blocks: ./check.sh green, the battery reads 28 of 28 killed, and nothing anywhere recorded that it was 41. mutants = N per battery, validated in load_battery, makes deleting a row a two-line edit that says so.

MEDIUM — the argument for keeping them was weaker than it first looked, and saying so changed which argument to make. The first draft argued the claim outlives its proof. It does not: re-deriving 41 mutants against a gate is an afternoon. What is unrecoverable is the reasoning about which mutants were worth writing — and the durable channel for that already existed. A per-mutant table can go in a retro.d/ fragment, and docs/RETROSPECTIVES.md has exactly one, against 93 commits that speak of a mutation pass. The channel existed, worked, and was used once. That is the case for a directory, and it is a better one than there was no channel.

MEDIUM — consolidation is a check that scattering cannot perform. Merging six scratchpad batteries into three files surfaced what no individual battery could: one battery entirely superseded by a later copy of itself, and two mutants byte-identical in file, old, new and kills under different names — the same edit counted twice in a report reading 29 killed.

MEDIUM — a directory nothing reads is the failure this repository keeps recording. A template version that meant different bytes in every commit; a verification table where 61 of 98 test paths did not resolve; a docs/GUIDE.md whose commands no longer ran. So tests/test_batteries.py reads them, in 0.05s. Its cost, stated rather than discovered later: a refactor that moves a mutated line turns ./check.sh red until the battery is repaired — measured at 2 repairs per 81 mutants per week, minutes each. The assertion names re-anchoring before deletion, because deleting a mutant to green a gate is the silent damage this arrangement invites.

MEDIUM — the corrections a review pass made to this file's own claims. Each was a number stated with more confidence than the data carried:

• "the single comment-bearing anchor in the set was the one that went ambiguous" — wrong on both halves. Three of the 81 anchors carry a comment; the one that went ambiguous carries none. One comment-bearing anchor did rot, in the superseded battery. The generalisation drawn from it — that the fragile shape is knowable in advance — went with it.
• The mcp battery was described as spanning six files. It spans seven; check.sh was in neither enumeration.
• The section for tools/fragments.py was labelled 0.28.0; the property shipped in 0.27.1.
• Its header was a comment that had migrated during consolidation, claiming "the two gates" and a control that had gone to the other file. A comment attached to the wrong mutant is silent, and the comments are the reason these files exist.
• A mutant named "ci.yml guards the gate with || true" appended && echo ok, which does not swallow the failure the name promises. The name is the only guard against an anchor that still resolves while the semantics moved, so a name that misdescribes its own edit disarms it.

MEDIUM — "regression asset" is the one thing this cannot be, and the README now says so. report() exits 0 when mutants SURVIVE, deliberately — "It is not a harness failure, so this exits 0 — read the rows." Nothing here changes that, and the committed gate never runs a battery: it checks that strings resolve. Anything CI-shaped built on top needs its own check on the survivor count. And the denominator belongs beside the claim: four batteries, four targets, all under tools/; no module under src/ has one, and no invariant in docs/INVARIANTS.md is covered. The covered files change 3–11 times a month, cli.py 58 and sync.py 40. A coverage index with a hidden denominator is the exact defect this repository keeps catching in its own gates.

LOW — --check-anchors reads the working tree, not HEAD, and the difference is the point. A run refuses an uncommitted target so git checkout <file> stays a safe recovery after a hard kill. Nothing is written in check mode, so inheriting that refusal would have bought nothing and disarmed the check in the one moment it is wanted. Two guards that look alike had different reasons, and copying the reason across would have made the second useless.

LOW — three ways the first draft of a "report every failure" contract reported fewer. The test-file refusal raised inside the per-battery loop, discarding the problems already collected and never reading the batteries after it — which is the run's behaviour, and the thing this mode is defined as not being. The claim set keyed on the battery path as typed, so one battery named twice under two spellings was a false double-claim. And stdout was flushed only on the problem path, so a refusal printed above the summaries it contradicted.

LOW — the flagship test passed when half of what it pins did not happen. The two stale mutants were named gone and also gone, and "gone: …" in stderr is satisfied by the also gone line alone. Reporting only the last problem passed a test whose docstring calls reporting every problem "the difference from a run that matters most". Two fixture names where one is a substring of the other is a silent half-assertion, and the same shape hides anywhere a test asserts on substrings it also chose.

LOW — the test helper that wrote both batteries to the same filename. battery() hard-coded battery.toml, so the first "two batteries" test built the second by calling it again and copying the result — silently overwriting the first. The overlap test passed anyway, because both files claimed the same target either way; the control — two batteries claiming different files — is what exposed it. A control earns its place by failing for a reason the pin cannot.

Extracting CLAUDE.md — four agents each edited correctly, and the set was wrong (20260823 13:52)

HIGH — every defect in this increment was a neighbourhood defect, and none was a content defect. Four agents wrote four destination sections in four files. Each one did its own job correctly: right heading, right anchor, every source fact carried, matching voice, no existing heading renamed. The review then found five defects, and all five were about what the surrounding text already claimed:

Where	What it still said
docs/RELEASING.md:3-5	"the traps … stay in CLAUDE.md" — one had just moved into this file
docs/RELEASING.md:7	"Extracted … 20260801 … Nothing was dropped" — a provenance note now silently covering content added 20260823
docs/INVARIANTS.md:6-9	described the file's two structures; there were now three
docs/README.md:83	the routing table still described the pre-extraction layout
docs/BUILDING.md:168	pointed at CLAUDE.md's text, which is now the compressed version

docs/README.md § Conventions has said audit the neighbourhood, not the diff since it was written. A per-file agent cannot follow that rule, because the neighbourhood is the thing outside its file — and the one file that most needed editing, the routing table, was assigned to nobody and so was the only docs/ page absent from the diff. If work is fanned out per file, something has to own the seams, and it is not any of the writers.

HIGH — the extraction nearly created a second home for a rule, which is the failure it exists to prevent. A section was written into RELEASING.md for naming unbuilt work — and docs/README.md § Conventions already owned that rule in full, including the same date, the same STATUS.md pointer and the same historical-records carve-out. Worse than duplication: the pointers formed a loop in which no file admitted README held it. README said the live names are in CLAUDE.md; the new section said the rule and the table live in CLAUDE.md; CLAUDE.md pointed at the new section. A reader following the pointer from CLAUDE.md would never reach the copy that actually owns the rule, and the two could drift with every command reporting success. The section was deleted and CLAUDE.md now points at README. Before writing a destination, grep for the fact — a relocation into a fact's existing home is a duplication, not a move.

MEDIUM — and checking that duplicate turned up a stale claim nobody was looking for. docs/README.md § Conventions still named the deep release as live unbuilt work. It left that table at 0.26.0 — its final cut — three releases ago, and CLAUDE.md's own table had been correct all along. The duplicate was not merely redundant; the two copies had already drifted, which is the concrete form of the harm "one fact, one home" is stated to prevent.

MEDIUM — four dead links, written while creating pointers whose entire purpose is to be followed. CLAUDE.md sits at the repository root, so a relative ](RELEASING.md#…) resolves to /RELEASING.md, which does not exist. Four were written that way. mkdocs build --strict cannot catch this class: CLAUDE.md is not part of the built site, so its outbound links are checked by nothing. The check that found them is three lines of Python — parse every ](path.md#anchor) out of the file, slug every heading in the target, compare — and it belongs in any future extraction.

LOW — the honest outcome is 220 lines, not 150. The guideline is ~150 and this file is at 220 after everything with a genuine home elsewhere was moved. Going further would mean deleting rules the user set, and rule 6 says explicitly that the guardrail is "the trigger to extract sections and leave pointers — not a hard cap that justifies deleting information." Reporting the shortfall is the correct outcome; compressing past it would have been the defect the rule warns about. What remains is dense by construction: a role rule, an ownership table, a naming table where every row is a breaking change, and four sections of user-set working mode.

The ungated link surface, and three instruments that lied (20260823 14:38)

HIGH — a regex cannot gate Markdown links, and both agents on this repo built one first. The coder's first scan reported 82 broken links inside docs/, a surface mkdocs --strict guarantees clean; it had blanked inline code with re.S, so one unbalanced backtick in a 2 616-backtick file swallowed every heading after it. Mine then reported 18 where the true number was 11 — twice, for two different reasons. Rendering answers the question directly: if a link is inside a code span the parser emits no <a> at all. No stripping, no heuristics, no reimplementation of block parsing. The final instrument is markdown.markdown() plus this repo's own _github_slugify, run through the same uv run --no-project --with-requirements requirements-docs.txt invocation make docs uses, so it cannot drift from the site.

HIGH — a control leg validates only the behaviours it exercises, and a green one is not a clean bill. The coder's control caught its re.S bug. Mine did not catch either of mine: 485 links across docs/, zero broken, green — while the instrument was wrong twice. docs/ contains no links to a directory, so Path.is_file() rejecting docs/graph/ and ../plans/ was never exercised; that produced six false positives. And docs/ has no indented code block containing a link, so treating one as prose was never exercised either. The second false positive produced a wrong edit before it was caught: retro.d/README.md:37 sits in a four-space block and is the README teaching fragment authors what to write — a bare #anchor, which resolves only once the fragment is spliced into docs/RETROSPECTIVES.md. It was repointed at ../docs/RETROSPECTIVES.md before the surrounding prose was read, turning an instructional example into one that teaches the wrong form. Reverted. Read what a line is for before fixing what a tool says about it.

MEDIUM — an adversarial pass that overturns nothing has said nothing. Re-verifying the 20260807 audit's 39 findings against c45ffa8 ran eleven agents plus an adversary per cluster, each prompted to refute every already fixed verdict, since that is the direction that silently drops a verified defect. The adversary overturned zero. Hand-checking all seven afterwards found one false: docs/DESIGN.md:711 asked to drop two phrases and only one had gone — a manifest and a template drift silently still stands while doctor.py prints the drift it denies. The adversary was prompted for exactly that shape, corrected in one place while an equivalent claim survives, and upheld it anyway. A zero from a checker is a claim about the checker. The same pass also corrected two predictions made from reading the code without reading the document: DESIGN.md:715 and KB-UPDATES.md:24 were called more wrong than before on the grounds that pnk upgrade had since shipped, when both had in fact been rewritten and are genuinely fixed.

MEDIUM — mkdocs --strict is a link gate, not a rendering gate. It resolved every link in docs/RETROSPECTIVES.md for weeks while one of its lines rendered as a broken run of <code> tags on the published site. The defect class — a backslash-escaped backtick inside a code span, which CommonMark does not honour — produces valid HTML that is simply not the HTML anyone intended, so nothing that checks structure can see it. Catching it needs a rendering comparison, which is a different check from link resolution and was found only because a link fix forced the line through a parser.

The ungated Markdown surface, and two checkers that were confidently wrong (20260823 14:36)

HIGH — a checker with no control leg reports plausible nonsense, and reads as authoritative. The first draft of tools/markdown_link_gate.py reported 82 broken links inside docs/ — a surface mkdocs build --strict guarantees is clean. The cause was a one-flag mistake: inline code spans were blanked with re.S, so in docs/ROADMAP.md (2616 backticks, zero fences) a single unbalanced backtick swallowed every heading after it. Nothing about the output looked wrong — missing anchors, real filenames, plausible slugs. It was caught only because the scan was run over a corpus whose correct answer is known independently of the checker. Run a new checker against a surface something else already guarantees, before believing anything it says about the surface nothing guarantees.

HIGH — a control leg validates only the behaviours its corpus exercises. A peer wrote an independent scanner over the same surface and got 18 where this one got 12. All six extras were its bug: it resolved targets with Path.is_file(), so every link to a directory failed — docs/graph/, ../plans/, ../retro.d/ — each correct as authored, since GitHub renders a directory link as a listing. Its control leg over docs/ stayed green throughout, because that corpus contains no directory links. So the two lessons are halves of one: a checker with no control reports nonsense, and a green control certifies only what it covered. Both instruments were wrong, in different directions, and each was found by something the other did not do.

HIGH — a false positive gets acted on, so the skips matter as much as the catches. The count was reported as twelve and was eleven. retro.d/README.md:37 sits in a four-space indented code block — neither this checker nor the peer's stripped those — and it is the README's own example teaching fragment authors which anchor form to write. The peer repointed it before reading what the block was for, turning an instructional example into one that teaches the wrong form, then reverted. Nobody argues with a false positive the way they argue with a false negative; they fix it, and the fix damages the document. A link checker's skip list is a correctness surface.

HIGH — when a cheap instrument is suspect, do not argue about it: render, and measure the gap. The right response to "a regex cannot parse Markdown" was not to defend the regex or to take the dependency, but to build the renderer once as an oracle and compare. Python-Markdown over all 114 tracked files emits 894 links; the extractor now finds every one of them and invents zero, the single difference being a bare <https://…> autolink that is external and never resolved anyway. That measurement is what licenses staying stdlib-only — ./check.sh and the CI job need no install and no network — and it is kept as a test that runs wherever the docs toolchain is present and skips with its reason where it is not. The argument was unresolvable; the measurement took one command.

MEDIUM — a false negative is the failure mode a link checker actually has. The second bug in this one forbade newlines in link text, which silently skipped every link whose text wraps — in a repository that wraps prose at 100 columns, and where CLAUDE.md wraps seven of them. A false positive gets argued with; a false negative is a clean bill nobody earned. The control leg cannot see it, because a control proves the absence of false positives only. The gate therefore counts the link syntax it saw and fails on anything it could not parse, rather than skipping it.

MEDIUM — the two documents most worth gating are the two the gate cannot fix. All eleven findings sat in planner-owned files (CHANGELOG.md, plans/, retro.d/README.md), so the gate was buildable and the defects it found were not — by the same ownership rule that makes the split work. Coordination, not escalation: the findings went over as a list with evidence and line numbers, and the fixes landed on the other side of the boundary while this branch built the gate.

LOW — six of the eleven were quotations, and that is a design constraint rather than noise. plans/20260807_2143-docs-audit-findings.md quotes other documents' links verbatim; they are correct relative to docs/ and dead relative to plans/, and correcting the path would falsify the quotation. A gate that flagged them could only be satisfied by corrupting the quote. Code-span a quoted link and it is inert — which the audit file already did in one place before anyone asked.

Landing beside a peer — the overlap tool answers a different question (20260823 14:57)

MEDIUM — a check named for a problem is not a check for that problem. tools/shared_file_overlap.py --fetch --strict reads as the collision check between concurrent sessions, and two sessions ran it that way for a full afternoon. It compares a branch to origin/main. It never looks at another branch, so it cannot see a peer at all: it reported none while a coder held 20260823_1424-markdown-link-gate. It was not wrong — the file intersection genuinely was empty — but it could not have been right, because it was answering "will this merge cleanly" and being read as "is anyone else in these files". The remedy is a comm -12 over the two branches' file lists, which is three lines and answers the actual question.

MEDIUM — the landing order can be forced, and asking your peer will not reveal it. The convention is that complete, gated work lands first and the other rebases, which frames order as something two sessions agree. Here it was determined by a gate. The coder's new tools/markdown_link_gate.py reported 11 broken links and exit non-zero against main, so its own ./check.sh could not go green until the planner's link fixes landed — its branch was unable to land, not merely second in a queue. Neither session knew. The coder was running the gate on its own tree, where it passed; the planner found it by copying the gate out of the peer's branch and running it against main and against its own worktree. Run the peer's gate, in both places. A peer's report of its own state is honest and still blind, because the failing configuration is the one it never runs.

LOW — the same window bites twice, so it is a property of the procedure and not of one tool. Both sessions' link checkers crashed with FileNotFoundError on a file git ls-files lists and the disk does not have. That state is not exotic: docs/RELEASING.md step 1 splices fragments and deletes them, and ./check.sh runs before the commit that removes them from the index. Any gate enumerating tracked files and opening each one meets it on every release. A tracked-but-absent file is a normal mid-release state, and the handling is a skip.

The gate that read everything except its own output (20260823 15:11)

tools/fragments.py --check had run in check.sh since 0.24.0. It parsed every pending fragment — category, slug, emptiness, a leading --- fence — and asserted nothing about the document --apply produces. Three malformed regions sat in CHANGELOG.md through every green run, and the one that names the class was found by reading a release precedent, not by any gate.

The shape is worth naming, because it is not a missing check — it is a checker pointed one step short of the thing it exists to protect. Every other gate here reads rows: mkdocs --strict resolves links, release_order_gate reads sequences, markdown_link_gate reads link targets. A duplicated heading is a property of adjacency, so all of them walk straight past two identical headings in a row. Nothing in this repository had ever read the assembled file at all.

The bug this increment shipped, and what caught it

The first draft anchored its fence detector at column zero. CHANGELOG.md contains no column-zero fence; its only fenced block is indented two spaces inside a bullet. So the scanner skipped nothing in the one document it most needs to read, and a changelog entry demonstrating a Markdown heading would have been refused as document structure.

It was caught by the test written for it, before landing — the false-positive test, not the true-positive one. Both were written in the same pass; only one of them found anything. A passing true-positive says nothing about what the checker is failing to look at. That is the argument for writing the over-firing case even when the under-firing case is the one the item describes: a gate refusing a correct document is the failure that gets acted on, and markdown_link_gate's own docstring records a peer rewriting an example into the wrong form to satisfy a false positive before disbelieving it.

What the false-positive probe found instead

Probing the - rule against Markdown a changelog author might plausibly write turned up a defect that has nothing to do with this gate. The 0.24.0 front-matter residue — --- / category: added / ---, spliced verbatim before check refused that shape — is not inert text. A --- after a text line underlines it into a setext H2, so each residue renders as a heading titled with its own metadata. There are five, not the three this module recorded: three in CHANGELOG.md under ## [0.24.0] and two in docs/RETROSPECTIVES.md, which the original count missed because it looked only at the stream the refusal was written for. The published site carried <h2 id="category-lesson">category: lesson</h2> twice — page, permalinks and search index. All five were removed at 9718aaa, in the same afternoon and by the agent that owns those documents.

Every instrument is green on it. mkdocs build --strict resolves links and a spurious heading is not a broken link; markdown_link_gate reads link targets; and this increment's own gate misses it, because it reads ATX headings and this is setext. Which is the same sentence as the item it closes, one level up: a defect that exists only in the assembled document is invisible to every instrument pointed somewhere else in it. The documents are the planner's to repair and a setext rule is a plan decision, so this increment recorded the measurement and changed no behaviour.

The mutation pass, and the survivor that was not mine

Twelve mutants, and the first run reported ten killed and two survived. Both survivors were real.

Mutant	First run	Why
the front-matter refusal no longer reaches --apply	SURVIVED	0.27.1's row, de-fanged by this increment. The document gate gave --apply a second reason to refuse that same fragment: a spliced --- is not a - list item either. Every assertion in the test stayed true through the mutant
the bullet rule reaches the free-form stream	SURVIVED	my own discriminating row. The fixture was built from ## headings and the bullet rule only ever reads a ###, so the scoping guard could be deleted with the test still green

The first is the one to remember. Adding a guard can silently un-pin a different guard, when both refuse the same input and the test between them asserts only that it was refused. Nothing about the front-matter check changed; a test that had killed its mutant for three releases stopped, because a second correct behaviour started covering for it. A test that cannot say which guard fired pins neither, the moment a second one exists. It now names the refusal.

The second says the ordinary thing about fixtures: a fixture that omits the construct the rule reads exercises nothing. The real docs/RETROSPECTIVES.md carries thirty-four ### headings, every one opening with prose — the fixture now carries one. Re-run: 12 killed, 0 survived.

What the adversarial pass found, and the shape all three share

Five lenses over the diff. Three real defects, none of them in the two rules the item decided — all three in the machinery around them, and every one reproduced end to end (written, fragments deleted, exit 0, follow-up --check green on the wreckage) before it was touched.

Defect	Why it was invisible
_merge_into_section spliced an entry inside a fenced code block	The checker skips fences; the splicer did not. A column-zero ``` holding ### Added was a heading to one and not the other
splice would take a quoted ## [Unreleased] as the insertion point	Same disagreement, one function up — it would bury every future release inside a code block
--apply was not atomic across streams	Refusing on the second stream wrote the first and deleted its fragments, then exited 1 saying "Nothing written, no fragment deleted"

The first two are one sentence: a checker and the code it checks must agree about what a heading is. Before this increment nothing read the document, so the splicer's fence-blindness had no second party and was merely latent — genuinely harmless, for eleven releases. Adding a gate that claims --apply cannot leave the document malformed is what converted a latent asymmetry into a false claim, and that is why closing it belonged in this change rather than in a new item.

The third is the plainer lesson and the one with teeth: a refusal message is an assertion about the world, and it was false. "Nothing written" was true of the stream that failed and false of the run. A release step is one step or none.

The check stopped being read-only, and that is a cost worth naming

The item's stated defect is that --check "asserts nothing about the result of --apply". The first draft read the file on disk — which asserts something about the result of the previous --apply, and nothing about the next one, the one the sentence names. So the narrow draft satisfied a weaker item. --check now validates the assembly the pending fragments would produce.

The evidence is the replay. Both instances the item cites came from a fragment whose body opens with its own ### Fixed. Run against the tree as it stood at each of those commits, the widened check exits 1 at the commit that added the fragment — at 0.6.0, whose duplicate was hand-repaired seven minutes after release, and at 0.28.3, twenty-two days before anyone noticed. Both exited 0 before, and the failure surfaced at release time — which is exactly when docs/RELEASING.md has somebody weighing whether to hand-edit a document to get a release out.

And the cost, which a peer named and this increment had not. --check is no longer a read-only validator: it simulates the write. So --check and --apply must agree about assembly forever, and a disagreement between them would be silent — --check green on an assembly --apply would never produce. That is this increment's own failure mode, reintroduced one layer up. It is not an argument against the widening; it is the argument for making the two the same code rather than two paths that agree: main calls prospective, the function --check validates through, and a test imports the module by path and compares the predicted bytes against what the real subprocess writes. A refactor that gives either path its own assembly turns it red.

Every new guard buys a new invariant. Three of this increment's four defects were the previous instance of that: a checker and a splicer that disagreed about what a heading is, harmless for eleven releases because nothing read the document, and false the moment something did.

The hole that was pinned rather than closed

The decided rule is adjacency, because that is the shape the evidence had. A heading that repeats after intervening entries is not caught, and there is now a test asserting so, naming the plan item. Widening the rule fails that test, which reopens the decision instead of quietly changing the gate. A named hole is a decision; an unnamed one is a discovery waiting to happen.

The assembled document nobody reads, and an extent computed after it was destroyed (20260823 16:18)

HIGH — a defect can exist only in the assembled document, where no instrument is pointed. Five blocks of --- / category: <x> / --- survived a splice into CHANGELOG.md and docs/RETROSPECTIVES.md. They are not inert: --- after a text line is a setext underline, not a thematic break, so each rendered as an H2 titled with its own metadata — and two of them have been served from the published site with live permalink anchors. Every gate was green. mkdocs build --strict resolves links and a spurious heading is not a broken link; tools/markdown_link_gate.py reads link targets; and the new fragments document-gate reads ATX headings, so setext is outside it by construction. Three instruments aimed at these two files, and the defect sat between all of them. The same shape as the item closed hours earlier — fragments.py --check validating what it reads and never what it writes — which is the tell that the class is the assembled document, not any one checker.

MEDIUM — a bottom-up edit destroys the markers a top-down extent depends on. The repair had to delete each residue and re-bullet the entry it preceded, so each entry's extent ran from the residue to the next structural marker. Deleting bottom-up is the standard way to keep earlier line numbers valid — and here it was wrong, because a later residue is an earlier entry's stop marker. The first pass deleted it, the extent then ran on into the following entry, and the result indented a top-level changelog entry into a nested sub-item of its predecessor. Nothing failed. No assertion fired; the residues were gone and the spurious headings were gone, so every check the repair had written for itself said yes. It was caught by reading the rendered structure afterwards and seeing one NESTED! where a BULLET belonged. Compute every extent against the untouched document first, then apply. And the older rule that would also have caught it: read the diff an edit produces, never its anchor.

LOW — the count was five and the record said three, because the record counted one stream. tools/fragments.py's docstring named three residues in CHANGELOG.md. There were also two in docs/RETROSPECTIVES.md, missed because the note was written from the refusal that motivated it, and that refusal is changelog-shaped. A count taken from the thing that prompted the search inherits the search's scope.

A closing note credited a fix that predated the defect, and the defect recurred twice (20260824 00:23)

tools/fragments.py spliced two consecutive ### Fixed headings into CHANGELOG.md at 0.6.0, and again at 0.28.3, twenty-two days later. Between the two, the item describing it was closed. The closing note named a mechanism that could not have fixed it, and the evidence for that was already in git log when the note was written.

When	What	Verified by
20260729 04:20	_merge_into_section introduced (0.3.0)	9f7ed5c, the only commit git log -S'_merge_into_section' returns
20260801 10:53	0.6.0 ships with two ### Fixed — three days later	68ca96b; two headings under ## [0.6.0] at that commit
20260801 11:00	hand-repaired, seven minutes after release	5920f41
20260803 22:21	item closed, crediting _merge_into_section	5bef91f, into plans/open-corrections.md
20260823 04:13	0.28.3 ships the identical defect	2f633a8

The closing row reads: "Fixed with a test (tests/test_fragments.py). _merge_into_section reuses an existing ### Category heading, bounded to the anchor's own section…" — true of that function, and irrelevant to the defect, because the row conflates two mechanisms:

what _merge_into_section fixes   the section's *prose* already carries the heading
what actually kept happening     a *fragment body* opens with its own `### Fixed`,
                                 which `render` then wraps in a second one

Both recurrences are the second mechanism, and it is checkable in one command:

$ git show 68ca96b^:changelog.d/fixed-two-frozen-yaml-behaviours.md | head -1
### Fixed
$ git show 2f633a8^:changelog.d/20260823_0233-fixed-published-versions-row-…md | head -1
### Fixed

Three instruments that reported the previous truth, in one exchange

The same afternoon produced three more, all of the same shape — an instrument reporting the previous truth is indistinguishable from one reporting the current truth, unless you know which it is. None of the three is exotic; all three are the everyday tool used the everyday way.

Instrument	What it actually answers	How it misled
git fetch then reading origin/main	what did I last hear from the remote	Reported a peer's landing as absent. origin/main is a local cache; only git ls-remote origin refs/heads/main asks the remote
git log -S <string> -- <path>	history of this path, stopping at renames	Returned only the rename commit for a plan file, dating a closure 20260804 instead of 20260803. In a repo that renamed every plan to YYYYMMDD_HHMM- retrospectively, every -S in plans/ inherits this; --follow or the pre-rename path is the answer
awk '…{print NR}' fileA fileB	cumulative record number across all files	Reported residues in docs/RETROSPECTIVES.md at lines 9718/9748 — the real lines plus CHANGELOG.md's 3957. FNR is the per-file number, and a line number carrying another file's length in it looks exactly like a line number

The third was mine, in a message asserting a peer's work had not landed. Both halves of that message were wrong, from two different instruments, in the same sentence — and the report read as confident because each number was real. A wrong number that was measured is more persuasive than a guess, which is what makes this class expensive.

The cheap defence is the same in all three: name what the instrument reads before trusting what it says. A cache is not a remote, a path is not a file's history, and NR is not a line number.

The lesson, which is about closing notes rather than about splicing

A wrong closing note is worse than an open item. An open item is a standing invitation to look; a closed one with a plausible mechanism beside it is a reason not to. It spends the evidence — the instance, the diagnosis, the commit that would have shown the dates — and buys nothing, and the next occurrence has to be rediscovered from scratch. This one was: 0.28.3's duplicate was found by reading a release precedent, not by any gate and not by the note.

The check that would have caught it costs one command. Every closing note names a mechanism; a mechanism has a commit; a commit has a date. Is the fix older than the defect it claims to fix? git log -S answers it in seconds, and here it would have returned a date three days on the wrong side. A fix that predates its defect is not a fix — it is a coincidence with a plausible name.

The generalisation, and it is the third instance of one shape in a single afternoon: a green signal is only evidence about the thing it actually measures. A closing note measures whether somebody was satisfied, not whether the defect is gone. A passing true-positive test measures the case it constructs, not what the checker fails to look at. And a test asserting that something was refused stops pinning why the moment a second guard can refuse it too — all three were found here, and all three had been green for weeks.

Two precedents that agree are not a pattern (20260825 00:11)

HIGH — a decision arrived pre-framed by two real precedents, and the framing was refuted by the three it did not mention. The question was whether the owed release was 0.31.0 or 0.30.1. It was handed over as a genuine fork with evidence on both sides: 0.27.0 was MINOR for a new tool file; 0.28.3 was PATCH for extending an existing one; 0.27.0's precedent is a new file, this is not. Every clause of that is true. It is also the two cases out of ten that agree with each other, and the criterion it implies — new file versus existing file — appears nowhere in the governing SemVer rule, which says MINOR = new skill, command, capability, or feature and PATCH = bug fix, doc update, or dependency/component change.

Measured across the ten tooling-only releases (those whose src/ diff is __init__.py alone):

Claim	Verdict
new tool file ⇒ MINOR	refuted. 0.22.2 shipped a brand-new tools/release_order_gate.py as PATCH; 0.27.2 shipped a brand-new tools/wheel_import_gate.py as PATCH, with an ### Added section
an ### Added section ⇒ MINOR	refuted. Seven PATCH releases carry one; 0.18.0 is a MINOR carrying none
existing tool ⇒ PATCH	4 of 4 — and the set is not vacuous, which is the check that mattered

The load-bearing check was whether that last row was an artefact of confounding. If all four had been ordinary bug fixes to a tool, the axis would have been "fixes are PATCH" and would have said nothing about adding a capability to an existing tool. It was not: 0.27.1 added a brand-new refusal to tools/fragments.py — this same file — and was cut PATCH under ### Fixed, and 0.28.1 did the same to release_order_gate.py. The one apparent counterexample, 0.29.0 putting --check-anchors on the existing mutate.py under a MINOR, is confounded: that release's headline was five new battery files and --check-anchors is not among its Added bullets.

The generalisation is about the shape of the offer, not about versioning. A framing assembled from two agreeing examples is the most persuasive form a wrong answer takes here, because no sentence in it is false. The defence is not scepticism about the examples — they survive checking — it is enumerating the population they were drawn from. The test set is the one you did not choose.

MEDIUM — the categorisation was the real question, and the implementer is the worst-placed judge of it. The fragment was named added-…, and changelog.d/README.md makes the filename authoritative for the assembler. But it was chosen by the session that had just built the thing, labelling its own increment — the position with the least distance on whether what it built is a feature. The entry was re-filed under ### Fixed on grounds independent of the version choice, so the reasoning does not close on itself: it closes an item from a corrections plan, its body is a defect narrative, and it is replayed against history to show it would have caught two past defects, which is a fix with a regression test. A PATCH carrying ### Added has happened once in the modern convention (0.27.2); this does not make it twice.

MEDIUM — %an cannot distinguish the agents in this repository, and the commit body can. All 1029 commits are authored Luca Gattoni <…>, committer identical. Measuring those fields and generalising to "git cannot answer who wrote this" was wrong, and it was wrong in the specific way this repository keeps rediscovering: the instrument was named after what it was hoped to answer rather than after what it reads. The trailers are not uniform — 20+ distinct Claude-Session ids and four Co-Authored-By models — and they are in %B, which no identity-field query touches.

git show -s --format='%B' <sha> | grep Claude-Session

That turns "was this closing note written by the session that did the work, or by a later one reconstructing it?" into a checkable question, which is precisely the class behind the two defects closed the day before: a closing note crediting a mechanism three days older than the defect, and a message asserting a peer's work had not landed. Both were written by someone reconstructing, and nothing in the repository could say so.

MEDIUM — a section heading is a status claim, and no gate reads it. A ### item in plans/20260731_1202-open-corrections.md sits under ## Live while its own body, 26 lines down, carries **CLOSED 20260824 00:35**. grep '^## ' — how anyone triages that file — returns Live and stops. It cost a real collision: a coder session read it as live and was minutes from rebuilding a landed increment. The convention it should follow already exists two files away — T6 and T8 in plans/20260804_1016-template-release.md carry · **CLOSED <date>, <disposition>** in the ### heading itself — so the repair is conformance rather than design. Not done yet, deliberately: the item is partial, not closed (the body-rule widening is explicitly untaken), so moving it into ## Closed — recorded so nobody reopens them would file an undecided design question under a heading promising the opposite.

LOW, and it recurred twice in one session — a gate is only a gate when its exit status is what the next command reads. make docs 2>&1 | tail -12 followed by echo "DOCS_EXIT=${PIPESTATUS[0]}" printed DOCS_EXIT= — empty. The value read was the pipeline's, not the build's. Both instances were caught and re-run bare, so what actually gated the release was ./check.sh and make docs redirected to a log with $? read directly. The rule was already written, in CLAUDE.md, and had been read that same session. A rule that is known and still broken twice in four hours is a rule whose failure mode is invisible at the moment of committing it, which is the argument for the redirect-and-read-$? form being the only form ever typed, rather than the one used when it seems to matter.

LOW — a count word and the list beside it drift independently. docs/STATUS.md held two instances of forty-seven and exactly one was wrong: the other sits inside 0.30.0's own published-verification entry, where it was accurate when written. A replace_all would have silently falsified a historical record. Same family as the | head truncation that shipped a broken anchor a day earlier — the edit whose anchor matches more than once is the one that needs the count taken first.

LOW — a peer renaming itself is indistinguishable from a peer dying. This session's name changed from pinakes-d1 to pinakes-7c mid-work. A coder session correctly observed that pinakes-d1 had left ListAgents, correctly inferred a usage-limit kill, and wrote a careful handover describing a release as stranded mid-cut — while it was still moving. Nothing was lost, because the branch had been committed and pushed at every step, which is the actual defence. But ListAgents cannot distinguish gone from renamed, so a peer's absence is evidence about the listing and not about the peer.

LOW, and found by a number that failed to move — tools/markdown_link_gate.py reads git ls-files, so a file that is written but not yet staged is invisible to it. A full ./check.sh run immediately after writing this very fragment reported 84 ungated file(s), 301 link(s) — byte-identical to the run before the fragment existed. Staging it and re-running gave 85. So the ordinary sequence — write a fragment, run the gate, commit — produces a green that says nothing whatsoever about the new file, and the first thing to actually read it is CI, after the push. It is the same sentence as the item this release is about, one directory over: a green signal is only evidence about the thing it actually measures, and here the thing measured is the index rather than the working tree. Nothing was wrong with the fragment; the gate simply had not been shown it. git add before the last gate run, not after — which is also the only ordering under which the gate's answer describes the commit about to be made.

HIGH — a sweep for one direction of error is blind to its mirror, and the mirror is where the confident mistakes live. A peer session swept plans/ for work filed as done that is not. Its single strongest-worded finding was the opposite kind and it was wrong: sidecar.py:125-127 really does accept a uniformly non-string-keyed mapping and silently coerce it, and Decision 19 really is marked shipped in 0.5.0 — but the decision's own sentence reads "Top level only", so a nested mapping is outside its scope by construction, and the behaviour is separately recorded at plans/20260731_1202-open-corrections.md:205 under ## Not to be fixed — recorded so nobody tries. Behaviour filed as never-to-be-done was read as undone.

## Not to be fixed is exactly as load-bearing as ## Closed, and strictly easier to miss: a sweep hunting false closure has no reason to open it, and a vocabulary grep has no token for it. The confidence came from having read the code, which was accurate; what decided the question was the decision's own sentence, which had not been read. Withdrawn by its author on being shown both lines.

HIGH — the filter that decides what gets examined is itself unexamined. That sweep classified each ## section LIVE / CLOSED / MIXED from its heading, and sent only the first two for a semantic read. Nine of twenty files were dropped unread, and at least one wrongly: plans/20260804_1844-decision-parent-child-arity.md's ## The decision was classified CLOSED while holding the file's only outstanding work — a measured ceiling required before G5's gate runs. A section named for a decision is not a section whose decision has been discharged.

Correction, 20260825 13:12 — the lesson holds, the example was wrong. The full re-read (X3) confirmed ## The decision does hold live work, so the finding above stands. But it is not the ceiling: that was measured the same day and is recorded far earlier in this file, under the parent-child arity entry — locate it by that heading, not by a line number — at 53.42 parent-child rows per chunk, 94.7% of stored edges, +113.4% index growth. The live item is requirement 3 — if the ceiling is alarming, the immediate-parent variant is the arm to measure — and no such arm exists (tools/graph_matrix.py LEGS has seven legs and none is one). The sweep that found a heading-level claim was wrong then made a body-level claim that was also wrong, and it survived into a retrospective, which is the same defect one layer down. The number worth carrying forward is therefore 11 of 20 files read, not the finding count; a report's coverage is a property of its filter, and a filter that reads headings inherits every defect that headings have. That is the same shape as the item three paragraphs up — one instance found and generalised — one level higher: the classifier looked at eleven files and silently declined nine.

MEDIUM — the stale-section-under-a-current-banner defect has a second, larger instance. plans/20260729_0256-links-and-graph.md's entire ## Baseline asserts a world that ended at 0.11.0, while the file's own banner at line 8 already says so. A reader going top-down meets "G3 is the next increment to build" (:69), "Do not start any of them" (:82), "One cut remains … it is blocked, not pending" (:103) and "G3 is the increment to pick up" (:124) before reaching the correction. Verified against the tree rather than against the plan: store.py:28 reads SCHEMA_VERSION: Final = 3 and the schema carries both nodes and edges, so G3, G5 and G6 all shipped. None of it matches any status vocabulary; a grep finds nothing here.

And the fourth class is now demonstrated rather than hypothesised — work silently overtaken. :642 states "Its exit criterion is not discharged by any test." It is: tests/test_manifest_compat.py::test_a_manifest_requiring_a_newer_pinakes_names_the_version exists and asserts exactly it. Something closed the requirement and never came back to say so, which no search for a status word can reach — the sentence is a claim about the test suite, and only the test suite can refute it.

The check that was not git — and the mutant that was not a test (20260825 00:14)

HIGH — a substring test stood in for git's ignore semantics, and it was wrong in both directions. pnk init decided whether a KB's .pinakes/ could reach a remote with ".pinakes/" not in gitignore.read_text(). Four failure modes, all reproduced by running rather than by reading:

.gitignore contains	git ignores .pinakes/?	init said
.pinakes (no slash)	yes	⚠️ warns	false positive
.pin*	yes	⚠️ warns	false positive
.pinakes/ then !.pinakes/	no	silent	false negative
#.pinakes/	no	silent	false negative, with a consequence

The two false positives train a user to disregard the warning. The false negatives are worse because they are silent, and the commented-out line is the cheapest way to reach one: comment it out to debug, re-run init, and the ledger and every transcript are tracked with nothing said.

The general shape: a cheap proxy for an authority, where the authority was one subprocess away. git already knows what git ignores — it reads nested ignore files, .git/info/exclude, the user's global excludes and a parent repository's rules, none of which any amount of reading one .gitignore can see. The proxy was not a simplification of the rule; it was a different rule that agreed with it on the common case.

MEDIUM — the measurement trap that makes this easy to get wrong twice. git check-ignore .pinakes returns not ignored for the pattern .pinakes/ whenever the directory does not exist on disk: a trailing slash only matches a path git can already see is a directory. At init time .pinakes/ has never been created, so the obvious probe answers "unprotected" for the very pattern init itself writes. A path inside the directory carries the directory in its own name and answers correctly either way. Anyone re-testing this by hand must mkdir .pinakes first, or a correct pattern will look broken.

MEDIUM — check-ignore exits 0 when any argument is ignored, not when all are. Probing three paths and reading the exit status would have called a .gitignore naming only the ledger full protection, while the index stayed tracked. The matched paths have to be counted.

HIGH — the survivor was the useful half of the mutation run, and it found redundant code rather than a missing test. Seven mutants; six killed. The survivor deleted an explicit startswith("#") comment-skip from the offline fallback, and every test stayed green. Not a coverage gap: "#.pinakes/".rstrip("/") is "#.pinakes", which already fails a whole-line equality test, so the skip never decided anything. The code claimed a care it did not exercise, and the docstring said so in as many words. The loop became one any(...), and the row now mutates the property that is really load-bearing — whole-line equality against ".pinakes" in text, which is the shipped defect. A SURVIVED row is a claim about a pair, and this time the wrong half was the code.

MEDIUM — the tests answered from the developer's machine. check-ignore honours a global core.excludesFile and a system config. A developer with .pinakes/ in their personal global ignore file — a sensible thing to have — turned three of these tests red and made two others pass for a reason the code had nothing to do with (measured: 3 failed, 9 passed). An autouse fixture points GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM at os.devnull. The production code still honours those files, because a global ignore really does protect the ledger; only the tests must not depend on which machine runs them. A test that shells out inherits the whole environment, and the environment is not part of the fixture unless it is made to be.

HIGH — the replacement's first draft was a regression, and only an adversarial pass found it. It probed three named files: index.db, ledger.jsonl, deep/transcript.json. A .gitignore carrying *.db and *.json — an ordinary thing to write — ignores all three, so the new check reported protected while index.db-wal stayed tracked, and in WAL mode that file holds megabytes of verbatim document text. The substring test being replaced warned there. The general shape is worth more than the instance: a check that answers about three filenames cannot answer a question about a directory, and the docstring's careful reasoning that every probe must match made the answer sound complete when the probe set was the incomplete part. A sound argument over an unstated domain — every clause true, the population unexamined, which is the same skeleton as the version framing that opened this session and as the status-word grep that swept plans/. Three instances in one night, in three unrelated places. The probes are now opaque random paths only a directory-covering rule can match.

MEDIUM — a reviewer's fix is a hypothesis exactly like the code it corrects. The suggestion was two opaque probes, flat and nested. Measured against ground truth — git add -A then git ls-files --cached, so the oracle never goes through check-ignore — that set still reports protected for .pinakes/* with !.pinakes/cache, which tracks the entire extraction cache. Four probes, two of them under cache/extract/ and deep/, is what passes all eleven configurations. The review was right about the defect and wrong about the fix, and only measurement separated those two — a distinction that disappears if a reviewer's remedy is applied because the reviewer was right about the problem.

MEDIUM — a hand-rolled fallback disagreed with the thing it was standing in for. Outside a repository the text scan read .pinakes/ followed by !.pinakes/ as protection — silently, on the very input the in-repo tests assert must warn — and warned about /.pinakes/, which git honours. Two definitions of one answer will diverge; the fallback now copies the .gitignore into a throwaway repository and runs the identical probes, and reading the file is reached only when there is no git at all.

MEDIUM — the increment had quietly widened its own trigger, and justified it with something false. The check was made to run whether the .gitignore was adopted or written, on the reasoning that an ancestor repository could negate what init wrote. It cannot: git resolves ignore rules by directory depth, so the file written in the KB wins over any ancestor. What the widening did reach was a path already in the index, where check-ignore reports not-ignored and the warning would tell the user to add a line their file already contains. Narrowed back, and the message that case produces was fixed rather than left: the printed remedy must never instruct an action that would not change the verdict, which the old text could not violate — under a substring test the string's presence was the verdict, so a redundant remedy was impossible. Asking git split "unprotected" from "the line is missing" into two facts and the message went on assuming they were one. A capability arrives with the failure modes it makes reachable, and they are invisible from inside the change that adds it. Scope creep that survives is usually scope creep with a plausible comment attached.

HIGH — two mutation survivors, and both were about the test rather than the code. The first deleted a startswith("#") comment-skip and nothing went red: "#.pinakes/".rstrip("/") already fails whole-line equality, so the skip decided nothing and the docstring claimed a care it did not exercise. The second was len(matched) > 0 surviving against test_ignoring_only_part_of_pinakes_is_not_protection — a test whose name is that assertion. Its input had been a .gitignore naming only the ledger, which discriminated while the probes were named files and stopped discriminating the moment they became opaque: no probe matches, so "any" and "all" agree. A test can stop pinning the property it is named for without changing a character, when what changed is the code it points at.

MEDIUM — the tests answered from the developer's machine. check-ignore honours a global core.excludesFile and a system config. A developer with .pinakes/ in their personal global ignore file — a sensible thing to have — turned three of these tests red and made two others pass for a reason the code had nothing to do with (measured: 3 failed, 9 passed). An autouse fixture points GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM at os.devnull. The production code still honours those files, because a global ignore really does protect the ledger; only the tests must not depend on which machine runs them. A test that shells out inherits the whole environment, and the environment is not part of the fixture unless it is made to be.

MEDIUM — shelling out brought six failure modes that had nothing to do with ignore rules. No timeout, so a wedged core.fsmonitor hook blocks init forever — after pinakes.toml is written, leaving a KB the next pnk init refuses as already a KB. An inherited GIT_DIR answering for an unrelated repository, which git exports to every hook it runs. Exit 1 read as an authoritative not ignored when git had in fact printed a warning. text=True decoding strictly inside subprocess.run, past except OSError. An inherited stdin. And a non-UTF-8 .gitignore aborting with a traceback. Each has a test. The cost of asking an authority is that you inherit its whole process model, not just its answer.

HIGH — 7/7 killed is a ratio with an unstated denominator, and it was read as an answer. The battery was green about the code that had been written, not about the code that should have been; the regression above sat outside every mutant in it, because the mutants were derived from the same understanding that produced the defect. It was reported as "7/7 killed, fragments written" and read as completeness by both the implementer who wrote it and the planner who received it. Neither asked what the battery covered. A green battery is a claim about the mutants someone thought of. The defence is not that someone should have been more careful — it is that a ratio whose denominator is unstated should read as a question rather than as an answer, and this one names the same skeleton as everything else here: a sound argument over an unexamined domain.

This is the first mutation battery under src/. Every other one covers tools/, which tools/batteries/README.md records as a limitation rather than a design — "no module under src/ has one, and no invariant in INVARIANTS is covered." The decision that keeps an index, a spend ledger and a set of verbatim questions off a remote is a reasonable place to stop being able to say that. Updating that sentence surfaced that it was already stale: it said "Four batteries, four primary targets" with five on disk. The paragraph that exists to prevent a hidden denominator had one.

Scope, deliberately held. Only the existing check's correctness changed — and its trigger, once widened, was put back. Whether pnk doctor should carry a recurring check, and whether it reports a warning or a note, is a decision reserved to the planner and the user, and this measurement changes it rather than settling it: a detector that misfires on .pinakes and .pin* is a poor candidate for firing on every doctor run, and a correct one is a much better one.

The diagnostic that printed nothing, and a review of a moving target (20260825 00:53)

HIGH — the fix for "a remedy that changes nothing" was an instruction that shows nothing. The message printed git check-ignore -v .pinakes/index.db. That command reports only positively matched paths, and the branch printing it exists precisely because the path is not matched, so it emitted nothing at all. One defect replaced by the same defect one step along, inside the commit whose subject line forbade the first.

The test is why it shipped, and the shape is general. It asserted the message contained the string git check-ignore -v. Asserting that a message mentions a command is not a test that the command helps. The replacement extracts the suggested line out of the printed warning, runs it, and asserts both that it produces output and that the output contains the negation the user has to remove. A test that reads a string can only ever verify the string.

MEDIUM — and the sentence describing the replacement overclaimed in turn, which is the fourth instance today. The proposed docs line said the command shows "the ! that defeats the rule". It shows the lines that name .pinakes — so when the defeating pattern is spelled some other way (!.pin*, !.pi?akes/, a broader !*, all of which reach the same branch and were verified to) the line that actually causes the problem is not printed. Caught by a reviewer running a case the author had not chosen, which is what caught the three before it. The typical case described as though it were the only case — the same shape as the probe set, the version framing and the status-word grep, in four unrelated places in one night.

Left deliberately, and named here so it is not rediscovered as a defect. A diagnostic that always shows the conflicting rule is a larger design question — cat -n .gitignore? a check-ignore -v against a path git does match? — and it belongs with the rest of the reserved gitignore work rather than in a release being held. The shipped command prints something real in every case the branch fires, which is the improvement; it is not guaranteed to print the cause.

MEDIUM — a review is a measurement with a timestamp. The adversarial pass took long enough that the branch moved five times under it, and reviewers measured different commits: several findings labelled critical were already fixed when they were written, and the adjudicator had to re-run two survivor claims to find both already killed. Its own caution is the lesson — a long review of a moving branch measures a tree nobody will ever ship — and the response is not to review faster but to re-verify every finding against the tree as it stands before acting on its severity.

MEDIUM — the same test-hygiene defect, one layer further out. An earlier fix in this line pointed GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM at os.devnull so a developer's global excludes could not decide the answer. It closed the config leak and left the ancestor repository open: with pytest --basetemp inside a checkout that ignores .pinakes/, three tests failed and two of them are named outside a repository while running inside one. The fixture covered the half that had been thought of — the same sentence as the mutation battery two hours earlier, about a different artefact.

MEDIUM — two things were in one list because they share a prefix. GIT_DIR and GIT_WORK_TREE redirect git at a different repository and must never be honoured here. GIT_CEILING_DIRECTORIES only limits upward discovery, so honouring it makes this check agree with the user's own git add and scrubbing it made the two disagree. Naming a constant for what its members look like rather than for what they do is how the wrong one gets in.

And a note on the review arrangement itself, because it worked in both directions. The planner amended a docs/CLI.md sentence specifically to stop it overclaiming what the code does — and approved the correction by comparing the prose to the implementer's description of the code rather than running the command. A description-versus-code check performed entirely on descriptions. Caught within the hour by an independent pass, which is the argument for having one.

Planning the exposure question — a check that cannot see the case it is for (20260825 07:55)

HIGH — the 0.30.2 gitignore detector reports protected for a repository that is committing the user's verbatim questions, and the reason is the fix that made it correct. The detector probes opaque random tokens under .pinakes/, chosen on 20260825 so that no realistic pattern could target them — which had been a real regression when three named files were covered by an ordinary *.db/*.json pair. But a path that does not exist cannot be in the index, and git check-ignore consults the index: on a tracked path it answers not ignored, and only --no-index answers ignored. So the probes can ask exactly one question — would a new file here be ignored? — and never the other one — is anything in there tracked right now?

Measured, not reasoned: a repository with .pinakes/{deep/op1.json,ledger.jsonl,index.db} committed, then a correct .gitignore added, still lists all three in git ls-files, still commits the edited transcript under git commit -a — and _ignored_by_git returns True.

The general shape, and it is the fourth instance in two days: a fix that removes a false positive can install a permanent blind spot, and nothing reports the trade. The opacity that made the detector right about patterns is exactly what makes it blind to the index. Neither property was written down, so the boundary was invisible to review.

MEDIUM — a correct .gitignore does not untrack an already-tracked file, and Pinakes says so nowhere. grep -rniE "rm --cached|untrack|already[ -]tracked|already in the index" over *.py *.md *.toml *.sh *.yaml returns only tools/template_drift_gate.py, tools/mutate.py and a retrospective about the wheel. This matters because init.py already states the rule it breaks — "the printed remedy must never instruct an action that would not change the verdict" — and for a tracked-and-unignored KB, "add this line" flips the verdict to protected while changing nothing about the exposure.

HIGH — a section heading is a status claim, and it produced the same collision twice in two days. plans/20260731_1202-open-corrections.md carries an item under ## Live whose body says CLOSED 20260824 00:35 twenty-six lines below its own **Decided.** line. On 20260824 a coder session read it as live and was about to rebuild a landed increment. On 20260825 a second, freshly-cleared coder session did the identical thing and offered to build it again. Both times the save was a peer message; neither time was it a gate. grep '^## ' returns Live and stops.

MEDIUM — sed -i '' "s/X\bY/" on macOS silently matches nothing, and a success message computed before the operation will not notice. Renumbering four colliding decision IDs, the loop printed "7 line(s) rewritten" — which was grep's pre-count, not sed's result. BSD sed does not implement \b. The file was unchanged and the log said otherwise. Reading the file back afterwards is what caught it; perl -pi -e is what fixed it. A count taken before an operation describes the input, never the outcome.

HIGH — the proposed fix for that blind spot failed in the blind spot's own direction, and two sessions caught it only by running it. The first draft specified git ls-files -- .pinakes, with a relative pathspec. Run from a subdirectory of a repository that is tracking three files under .pinakes/, it returns rc=0 with zero rows — and a genuinely clean repository returns rc=0 with zero rows as well. pnk init can be run from anywhere, so the two states are reachable, identical at runtime, and the wrong one reads as safe. The absolute pathspec returns three rows from the same directory. A check that answers clean because it was asked the wrong question is exactly what the increment it corrects existed to remove, so the fix has to make the pathspec structural — build it from the resolved root, and test it from a subdirectory, because the failing case is invisible from the root.

MEDIUM — a remedy whose steps are right and whose order is wrong fails silently. In a repository with no ignore line, git rm -r --cached .pinakes followed by an ordinary git add -A puts the file straight back in the index (measured: one file, immediately). The ignore line has to come first. The reverse order looks like it worked and reverts on the user's next add.

HIGH — knowing a hazard is not what avoids it; running the command bare is. The boundary table above was first read through ... 2>&1 | head -3, which reported the outside-a-repository case as rc=0 instead of 128 — the pipe's status, not git's. This repository had already recorded that exact hazard, twice, and the session that walked into it had read the entry that morning, within the hour. The entry is therefore restated in its stronger form: the rule does not fire on recognition. The only thing that separates a real exit status from a plausible one is running the gate bare and reading $?.

HIGH — the plan wrote a constraint from a measurement taken in the wrong place. § X1 required the tracked-check's pathspec to be absolute and prescribed the test that would prove it: "run it from a subdirectory — the case that fails is invisible from the root." The property is real. The prescribed test cannot fail, because _ask_git passes cwd=root to subprocess.run (init.py:94) and every call site passes root — so the process cwd never reaches git and a relative .pinakes resolves against root regardless. The reproduction behind the constraint was a shell fact, run with cwd=<subdir>, a state this code never produces. It was promoted to a claim about a code path without anyone opening that code path's cwd.

The domain nobody examined was the instrument. Every other item here is a claim whose population went unstated; this one is a test whose ability to observe anything went unstated. The argument for the property was sound and stays sound — the absolute form survives as defence. What was wrong was the belief that a named test would pin it. A prescribed test is a claim about a failing test, and whoever prescribes it owes the same evidence as whoever writes it. The increment's own retrospective carries the mutation run that settled it, and the finding is recorded there rather than here.

MEDIUM — the planner produced the day's own lesson while writing the plan about it. I told a coder that 36 unrowed test_fragments.py tests were a gap to fill. It counted the population instead: 2023 tests defined, 1106 named in docs/VERIFICATION.md, 917 with no row — and the document's own scope section says it maps promises to tests, naming six modules that predate the table. A gap of 917 cannot all be holes. The coder declined, correctly, and cited a peer message is coordination, never permission. Two honest counts of that same population differed by twelve (1106/917 against 1094/929) purely because the matching rule differed, and neither number carries its rule.

LOW — zsh does not word-split unquoted parameter expansions, so set -- $pair inside a for loop passes one argument, not two. The anchor assertion caught it before any write. Worth keeping only because the assertion is the reason: the guard that looked like ceremony was the thing that turned a silent no-op into a stop.

X1 — building the tracked-KB check, and an instrument that could not fail (20260825 08:16)

HIGH — the specification asked for a test that cannot fail, and only writing it revealed that. The plan bound the implementation to three constraints, the first being "the function takes the resolved KB root and builds the absolute pathspec itself… its test must run it from a subdirectory — the case that fails is invisible from the root." That constraint was written from a real, reproduced measurement: git ls-files -- .pinakes run from a subdirectory exits 0 with no rows against a repository tracking three files. Two sessions reproduced it independently.

It is still not reachable through this code, and the test it prescribes proves nothing. _ask_git passes cwd=root to subprocess.run, so the process cwd never reaches git: a bare .pinakes resolves against root and returns the identical verdict from a subdirectory, and with a relative root. Measured rather than argued — with the pathspec mutated to ".pinakes", 68 of the 69 tests in tests/test_init.py pass. A behavioural subdirectory test would have been green against the exact form it existed to forbid.

So the constraint moved to where it can fail. The pathspec is built by its own function and the test asserts the pathspec — :(literal) prefix, absolute, ending in .pinakes — which is the one test the relative form kills. The absolute-and-literal form stays, because correctness of the relative form depends entirely on _ask_git keeping cwd=root, a coupling no caller can see, and because a KB root may legally contain * or [. It is defensive rather than a bug fix, and the docstring says so instead of implying coverage it does not have.

The general form is worth more than the instance. "Pinned by test X" is a claim about a failing test was already the rule here. This adds the case the rule does not obviously cover: a constraint can name a real property, be derived from a real measurement, and still prescribe an instrument that cannot observe it. The argument was sound; the domain it quantified over — this call path, rather than the shell where the measurement was taken — was never examined. Neither session checked it. It surfaced only because the implementer refused to write test_…_from_a_subdirectory without first watching it go red.

A second gap the same discipline found. pinakes_tracked is a bool, so None and False collapse into one field and no test through init can tell them apart — an implementation returning False outside a repository (exit 128, fatal: on stderr) passed every test written. The constraint that unknown must never read as clean is now asserted against _tracked_by_git directly. A field's type can erase a distinction the specification depends on.

And the pipe hazard fired again, on the person who had just written it up. The full gate was run as ./check.sh > log 2>&1; echo "EXIT=$?" | tee -a log; tail -20 log. The harness reported exit code 0; the gate had exited 1, on two reportPrivateUsage errors. The $? was captured correctly and then discarded by the commands after it — a gate is only a gate when its exit status is what the next command reads, missed by someone who had recorded that sentence twice the same evening. Recognition is not the mechanism. Writing the status to its own file, and reading that file, is. Three instances of one class in one evening, each caught by running something rather than by reading it — and this one caught only because the log was read after the notification said the opposite.

MEDIUM — and the fourth instance of this increment's own shape: a CI watcher that was green for a reason nobody had checked. Its exit condition was "all visible runs are complete". That is unsound in general — with staggered run creation it exits before the remaining runs exist and reports success for a run that was never created. Here it gave the right answer, because exactly one run was ever going to be created: docs is path-filtered to docs/**, mkdocs.yml, mkdocs_hooks.py, requirements-docs.txt and its own workflow, and this increment touched none of them. But that fact was established after the watcher was armed, not before. The instrument was sound by luck over a domain its author had not examined — written an hour after the same author wrote that the instrument is the part nobody checks.

A second watcher, on the planner's side, failed the same way and was not saved by luck: gh run list --commit <short-sha> returns empty, so an until [ ... = "completed" ] loop could never terminate and would have polled in silence until timeout. --commit requires the full sha. Silence from a watcher is indistinguishable from still running, which is why both were rewritten to emit on every terminal non-success state rather than to wait for a success that may never arrive.

S2 — A census is not a plan (20260825 18:55)

HIGH — two mutants survived 267 tests because every assertion counted actions instead of ordering them. pairing.pair() returns a list of actions, and describe() — the helper the whole test file asserts through — reduces it to a census by kind. When the same-path branch stops emitting a SoftDelete, the vanished-path loop re-emits one further down the list, so the wrong plan and the right plan have an identical census: same actions, same counts, different meaning. documents.path is UNIQUE, so the order is the entire behaviour — an Adopt applied while the row it replaces is still active raises IntegrityError instead of replacing it. One of the two survived the entire suite. The generalisable form: when a function returns a sequence, a test that asserts its contents has not asserted the sequence, and a helper that summarises is where that blindness hides. The fix was a named property (retires_before_adopting) rather than a positional assert kinds == [...], so it states what must hold rather than what happened to be produced.

A SURVIVED row is a claim about a pair, and here the wrong half was the selector. The inverted-guard mutant was killed by five tests — just not the one the battery named beside it. Re-running it against the whole suite rather than the selector is what showed that, and it took one command. The battery's own README already says this ("either half of which can be wrong"); what this adds is the cheap procedure: before believing a survivor, run the mutant against everything, not against its selector. A survivor that nothing kills and a survivor whose selector is misaddressed need different fixes, and they are indistinguishable from the battery's output alone.

MEDIUM — the first discriminator was wrong, and a test falsified it rather than a reviewer. The doctor check was designed, reviewed and specified as "a retired row whose file and matching sidecar are both still on disk". That rule misses the state S2 is named for. When a sidecar's id changes at a path, the row that gets retired keeps its own old path, while the document that has become unfindable is the one whose sidecar now claims that id somewhere else — so a path-based rule reports the wrong document and misses the real one. What found it was writing a test for an adjacent case (a path reused by a different document) and watching it fail for a reason I had not predicted. The rule is now keyed on the retired id. Worth stating because the wrong rule had already survived a careful adversarial review: a specification written from the mechanism can be wrong in the same direction as the code, and only a case neither party considered separates them.

MEDIUM — running corrected a confident inference, and the correction changed the build order. From the pure function I inferred that a rename chain silently re-mints a published id — a Mint for a document whose sidecar already carried one, which would breach ULID permanence. Running it showed the crash preempts the mint: on main the first adoption raises before the Mint is ever reached. The defect is latent rather than shipped. That is not a smaller finding, it is a different one, and it carries an ordering constraint nobody had written down: fixing the rename collision without this fix first would unmask the re-mint. The inference was sound and the conclusion was wrong, which is this repository's most-recorded failure shape and was reproduced here in the space of ten minutes.

LOW — a condition that cannot be false is worse than no condition. The first draft of the guard read claimed is not None and claimed.document_path != path. The second term can never be false where it is evaluated: the sidecar beside that path is the one that disagrees, so the sidecar claiming the row's id is necessarily a different one. It would have shipped as an unkillable mutant — a line no test can pin, which reads as care and provides none.

MEDIUM — I measured the check as free, and it is not. It costs 12–17%. The first measurement said pnk doctor went 0.822s → 0.818s over 2000 documents and I wrote "free, within noise" into a fragment. An adversarial reviewer measured +12% on the same corpus, which sent me back to it. They were right. Re-measured alternating branch and main against one KB rather than running each side once in its own worktree: main 0.7099s / 0.6700s, branch 0.7959s / 0.7823s — +12.1% and +16.8% — and walk_document_paths alone is 49 ms over 1995 documents. The added work is real, it is the glob, and it is paid only when the KB has retired rows (otherwise the check returns before walking).

The method is the finding. Two sequential measurements taken in two worktrees cannot resolve a 10% difference on a machine that drifts by more than that between them — my "before" run happened to land slow and my "after" fast, and the difference I was trying to detect was smaller than the drift I did not control for. Interleaving costs nothing and would have shown it immediately. A/B by alternation, never A-then-B, whenever the effect is smaller than the noise you have not measured.

What survives is the comparison that actually mattered: the attempt this replaces measured 2.25x (0.746s → 1.682s) and this is about 1.15x, because the expensive question was never the check — it was a SHA-256 over the whole corpus and a second ruamel parse of every sidecar _sidecars() had already parsed. Reusing that parse and asking the walk for paths alone is what the ratio buys. "Cheaper by a factor of eight" was the honest claim; "free" was not, and nobody would have caught it if a reviewer had not run the number I had already published.

And the exit code is still not a validity signal, demonstrated while verifying this. Running the real pnk doctor against a healthy 2000-document scratch KB exits 1 — for embedding and reranker, because the fake backend is registered inside pytest and not in a plain CLI process. Two sessions built measurement harnesses on that exit code in one day; this is a third sighting inside an hour of deliberately watching for it. Assert on the named check row, never on the exit code.

HIGH — the fix introduced the exact shape it was removing, and only an adversarial pass saw it. Guarding the branch where a sidecar disagrees with the row left the ordinary branch alone, so a sidecar moved from one document onto another produced RefreshMetadata(X, a.md) beside Adopt(X, b.md): the same-path loop trusted the index row because nothing beside a.md contradicted it, and the adoption loop followed the sidecar that had walked away. One id, two paths, one plan. Measured on both sides: origin/main raises IntegrityError and keeps both rows; this branch exited 0, moved the row to b.md, and left a.md on disk with no row at all — indexed yesterday, unfindable today, nothing recorded. A loud failure turned into a silent one is the precise regression that condemned the previous attempt at this fix, and this increment reproduced it while carrying a commit message about not doing so. Two things made it findable: an independent reviewer that ran origin/main as a control rather than reasoning about it, and a UNIQUE constraint that had been the only thing noticing — by crashing. The lesson is not "guard the other branch". It is that a guard written for the case you are thinking about is not a property; the repair is places_each_id_once(), asserted over every shape the file exercises, because a helper asserted case by case is a helper nobody runs on the case they did not think of.

MEDIUM — a property helper that did not implement its own docstring. retires_before_adopting() was written to assert "no Adopt may land on a path this plan retires later", and built its index of retirements with reversed(list(enumerate(...))) — so for a path retired twice it kept the earliest position, and a plan that retires a path, adopts onto it, then retires it again passed. It was doing real work (it is the only thing that kills two of the battery's mutants) and it was still wrong about the sentence above it. A named property is worth more than an inline assertion and is also a second place to be wrong; the reviewer read the implementation rather than the name, which is the only way that class is ever caught.

MEDIUM — the count I published was scoped and I called it complete. This file said a mutant "survived the entire suite" and the battery said "the whole suite of 267 tests". 267 was pytest tests/test_pairing.py tests/test_sync.py tests/test_doctor.py — the three files covering the code — while the suite is 2167. Caught when a planner asked to publish the number in a document whose whole subject is not publishing counts nobody has run. Re-measured properly, at the exact commit where the survival was observed, with the mutant applied and the full suite run: 2165 passed, 0 failed. The claim was true and the denominator was invented, which is the same defect one level up from the one the paragraph was describing. State the population, not just the number — and when a scoped run is what you have, name the scope.

HIGH — I reported a defect as open after my own later commits had fixed it, and a planner recorded it as scheduled work. S17 (a rename that frees a path a new document then takes) was measured early, against origin/main and against my branch as it then stood; both failed, so I filed it as pre-existing and unfixed. It was pre-existing. But the moved-sidecar guard that came out of the second adversarial pass — four commits later — fixes it as a side effect, because S17's mechanism is the same one id at two paths shape. Re-measured with a control: at 03e6f86 the walk records SidecarError: … appeared after the walk had already read this directory and the renamed document is never indexed; at 3876b57 it reports 1 indexed, 1 renamed and the document is active.

The error is not the measurement, it is the report: a finding taken against a moving branch has a sha, and mine did not carry one. It was true when taken and false four commits later, and nothing in the report let either of us see that. It had already been written into a plan as a two-part fix, which is one message from a fresh session rebuilding something that works — the failure this repository's own CLAUDE.md records happening twice in two days. Stamp every cross-session finding with the commit it was measured at, and re-measure before anyone schedules it.

And the repair for the claim itself is a test, not a corrected sentence. "S17 is fixed" written in a plan is a fact that decays silently; test_a_rename_that_frees_a_path_a_new_document_then_takes is the same fact that goes red if it stops being true. Confirmed red at 03e6f86 before being believed.

LOW — and the same disease in a docstring I had just written. _retired_documents claimed the pre-commit hook's sidecar "cannot trip it: nothing the hook does retires anything", and the changelog fragment repeated it. A reviewer measured otherwise: at a reused path — a new document at a name some earlier document held — sync --sidecars-only does reach rule (2). The report there is true (the document has no row and pnk search cannot see it) and the shipped hook pair never leaves that state, since post-commit indexes inside the same commit. So the check is right and the sentence was wrong, which is the more embarrassing half: an absolute claim written to explain why a design is safe, in the same paragraph that had already earned the right to a narrower one.

Taking eight decisions into the documents — what the writing found (20260825 18:55)

The decisions were taken by the user at 20260825 18:16. This is what surfaced while writing them into the files, and every item is something the deciding pass did not know.

HIGH — a correction about false precision was itself a timezone error, and it nearly propagated. The decision record listed three defects in docs/VERIFICATION.md's scope sentence. Two were already resolved before the pass began: the 923→890 row count was corrected earlier the same day, and the "impossible" provenance stamp is not impossible — c23359f is 2026-08-25T13:42:55+01:00, i.e. 12:42:55 UTC, so the 12:49 UTC measurement is seven minutes after it. An adversary read a local timestamp as UTC. Had the planner trusted the list rather than checking it, it would have "fixed" a correct stamp into a wrong one — a repair that manufactures the defect it repairs. Checked with git show -s --format=%ad --date=iso-local, not by reading.

HIGH — the two numbers in one sentence rotted the same way, and the earlier correction pass fixed only one of them. The same sentence carried "890 rows" (corrected) and "62 of the 67 test modules" (left). The module figure was wrong on both halves: there are 74 modules and 63 carry a row. A sentence with two numbers of one class is a sentence where correcting one and stopping is the likely failure — the reader's eye treats the sentence as handled.

HIGH — the obvious way to count that sentence reports 74 of 74 and finds nothing. Counting "modules named in the document" by testing whether each filename appears in the text is wrong here, because the preamble itself lists the eleven unnamed modules by name. The measurement instrument is contaminated by the prose it measures. The gate's own REFERENCE regex — which matches only a tests/x.py::y reference — gives 63. This was hit live, mid-pass, and the wrong number was on screen before the right one.

HIGH — the bounded audit found a cluster, not an instance, and the cause was structural. D-34 was licensed on one unrowed promise found by sampling. Auditing a single module found 14 of tests/test_serve.py's 31 tests unrowed, two of them security boundaries: the MCP path-refusal (../../etc/passwd) and the labelling of retrieved text as evidence rather than instruction. The cause was not neglect — the server's rows lived under the links release and page citations, and no section owned the server boundary. A missing section is invisible to every gate and to every reader who checks whether a row exists. Fixed by adding the section, not by adding rows to the wrong ones.

MEDIUM — a tripwire whose condition was already met on the day it was armed. The staged-channel-gates clause said "two entries exist and are deliberate… if a third appears, that is the drift this rule exists to catch." docs/DESIGN.md had carried a third entry of the same shape since five days before the clause was written. It reported drift that was not drift, and cost two separate readers a full pass. A rule that counts instances rots; a rule that names a class does not. It now forbids a class — no plan, no increment, no numbered item, no version number — and says index, routing and naming entries are permitted and uncounted.

MEDIUM — a count went stale eleven lines from a row that already carried the right one. docs/README.md said the sweep held "fifteen confirmed defects" in its prose and "Sixteen defects" in the table below it. Both were written the same day. Proximity is not consistency, and the same file is where a routing table already carries the instruction "this row deliberately states no count" — a remedy that existed one row away and was not applied.

HIGH, and the worst of these — a gate was cited as evidence for a property it cannot see. A blockquote annotating a table row was written inside the table, which renders it as a table cell. The repair moved it after the last row with no blank line, so Python-Markdown's tables extension swallowed it as one more <tr> — a cell whose text begins with a literal >. mkdocs build --strict exits 0 on the broken form and on the fixed one alike, so "make docs EXIT=0", offered in a commit message as proof the repair worked, certified nothing about the repair. The generalisation is the same one this project already applies to tests and to mutation batteries, and it had not been carried across to gates: make the gate red on purpose first. If it cannot fail on the broken form, it is not evidence for the fixed one. It is also the same failure as check.sh | tail reporting tail's exit status — a green signal that is not a function of the property anyone cares about — and the two were hit by two different sessions within an hour.

LOW, and it cuts against this pass — the writing changed the number the writing states. The scope sentence records the table's row count. Adding fourteen rows moved it from 890 to 904, so a count copied from the parent commit would have been falsified by the edit that copied it. Stated in the file rather than silently re-measured, because it is the third time in one day a change here invalidated a pointer to itself.

Reconciling the queues — two registers of one fact, and an owner with nowhere to queue (20260826 06:32)

The increment was meant to close a scheduling gap CLAUDE.md names in bold. It found the gap was the smaller half.

HIGH — a file that holds two registers of the same facts will diverge, and no gate can see it. plans/20260825_1252-plans-sweep-findings.md carries a 27-row Actionable table with Status, Blocked-on and Owner columns, and a § Open questions list of thirteen bullets. They describe the same decisions. The 20260825 18:16–18:41 pass updated the bullets, inline, carefully — and never touched the table. Twelve of the 27 rows stopped describing the tree — eleven of them stating LIVE (eight) or UNCLEAR (three) for work that was built, answered, declined, deferred or ruled that evening. This is the repository's own recorded failure — a ## heading is a status claim and nothing gates it — moved one level in, into a table cell, where reading the item's body does not help because the body is the cell. The fix is not to maintain both. It is to say which one wins: a dated snapshot carries a disposition, a ## Build order carries the queue, and where they disagree the build order is right by construction.

HIGH — an owner is not a schedule, and the difference is measured in weeks. Three decided items had an owner and no queue position anywhere: the _toml.py unknown-key remedy, the paid re-extraction loop's trigger, and the G5 gate re-run — which sat that way for 21 days while every sweep that read headings passed over it. The seam is structural: an item whose owning plan has no ## Build order has nowhere to be queued, and docs/README.md routes to files, not to work. Parking them in one named section beat both alternatives — a fourth register would repeat the defect above, and a build order per plan makes a coder read four files to learn what is next, which is how the seam formed.

HIGH — the top of the coder's queue was a claim nobody had re-checked, and checking it took nine minutes. CLAUDE.md said S16 was "still live, reproduced on main 20260826". Nothing in the sweep plan supported that date — the only run on record is 32442db, 20260825 18:18, which is before S2's fix landed at 04:06 the next morning. And S2's fix is known to have cured S17 as a side effect, so "does it also cure S16?" was a live question that no document had asked. Reproduced end to end against a src/ tree byte-identical to origin/main: it is still live, all three failures intact. The finding is not that the claim was false — it happened to be true — but that nobody could have known, and it was written as though someone did. A defect recorded against a moving tree needs its sha; S17 was recorded the same way and had already been fixed.

MEDIUM — reading names instead of targets put a false claim in a file that exists to state a denominator honestly. tools/batteries/README.md named src/pinakes/cli.py among "the two highest-churn modules … [that] still have none". src-pinakes-init.toml mutates it twice. The same error hides more: tools-mcp_handshake_gate.toml reaches seven files including Makefile, check.sh, pyproject.toml and both CI workflows. A battery's name is not its coverage, and the question has a one-line answer nobody was asking — grep -h 'file = ' tools/batteries/*.toml | sort -u. Its own gate could not catch it: tests/test_batteries.py forces a battery whose stem does not begin tools- to be named in the README, and checks nothing about what the batteries reach.

MEDIUM — I invented a duration inside a pass about unmeasured claims. I wrote "fourteen hours" for how long the table said S2 · LIVE, four times across four files, and never computed it. That count is not re-derivable from git and this sentence is the only record of it — the error was corrected before the first commit, so history contains this description and not one instance. Stated here rather than left to look like a measurement: it is a report of a working tree nobody else can inspect, which is the weakest kind of claim this repository accepts and it is accepted only because a retrospective has no other way to describe what it caught. S2 landed at 04:06 and the pass ran at 06:19: two hours. The eleven decision rows were the twelve-hour ones. Two different numbers had been collapsed into one invented figure that flattered the finding. Caught only because the timestamps had to be re-derived for an unrelated reason.

MEDIUM — and the reason they had to be re-derived: --date=format-local where the rule says UTC. Local here is UTC+1, so 3876b57 went into three files as 05:06 when it is 04:06 UTC. The retrospective fragment written twelve hours earlier records an adversary making the same substitution in the other direction, and says it "would have 'fixed' a correct stamp into a wrong one". Reading that fragment did not prevent repeating it, which is worth more than the fix: the guard has to be in the command, not in the memory of a lesson. git log takes --date=format-local and TZ=UTC git log gives the rule's answer; nothing warns you which you typed.

LOW — main moved twice during the pass, and once it falsified a row as it was being written. A branch recorded as unowned — its author's session ended, and a direct message to the only other candidate failed HTTP 409 — landed while that row was being drafted. A peer said so; the claim was verified against git ls-remote rather than relayed, and the row was deleted before it was committed. The rule that saved it is the cheap one: verify at the moment of landing, not at the start.

make release-check — a gate that could not fail, and what that says about the others (20260826 06:38)

HIGH — the target had a help string, a docs/RELEASING.md step, a CLAUDE.md rule and a docs/STATUS.md sentence, and no comparison anywhere in its recipe. Four documents described a check; the recipe printed three lines and exited 0. Nothing was wrong with any single document — each was written by someone who had read the others. The failure is that a target's documentation and a target's exit status are separate artifacts, and only one of them is executable. The generalisable form: a check named in four places and implemented in none is indistinguishable, from every document, from a check that works. It survived from the target's creation to 20260826, and what found it was neither a gate nor a reader of the Makefile — it was a planner building an argument that rested on the phrase "the tree make release-check was run against" and going to read what that meant.

The remedy is not "test the gate" — it is make help and the recipe pinned in one test. The ## string is what a release operator reads and it is not an argument to the recipe, so the two can disagree silently forever. test_the_release_check_help_string_and_its_recipe_are_pinned_together holds them together. Two of this increment's mutants are the same idea one layer down: commenting the recipe out (make hands \t#… to /bin/sh, which exits 0) and a single leading - (make discards that line's exit status). Both restore "the target cannot fail" in one character, and both now die.

MEDIUM — a KILLED row is not self-validating, and this is the mirror of the survivor rule the batteries README already states. The mutant for "an annotated tag with an empty message passes" was written as if not …: → if False:. It reported KILLED. It was killed by an IndexError traceback out of splitlines()[0], not by the exit code — because removing that branch does not make the empty-message case pass, it makes the tool crash. So the row was a true statement about the wrong property: the tests notice a broken tool, not the tests notice an unannotated tag being waved through. Rewritten as an early return 0, it dies on assert 0 == 1. Read what killed a mutant, not that something did — mutate.py prints the assertion, and it is the only place the distinction is visible.

MEDIUM — the increment's own test of its own gate had the escape hatch the gate exists to remove. Nine of the eleven tests supply --repo and --expect-version, so the defaults are a region no fixture reaches; the one test covering them read assert str(ROOT) in output or result.returncode == 0, and on the green branch the or carried it and it could not fail. An unfailable assertion inside the increment whose entire subject is an unfailable check. A test seam does not only hide the real path — it concentrates the temptation to write a weak assertion there, because that is where a strong one is awkward. The repair was two assertions that can fail plus two mutants (REPO repointed, the default version hard-coded to 0.0.0) that are invisible to every other test in the file.

LOW, and named rather than closed: the gate never asks whether HEAD is on the remote's default branch. A tag on an unpushed commit passes all four legs, and pushing it hands the publishing workflow bytes that are on no branch. Every offline form of the check is wrong — a remote-tracking ref is as stale as the last fetch, and the remote's tip may be an object this clone does not have — and the correct form has to fetch, which makes a release gate mutate local state and go red for being behind, a state that is not a publish hazard. Left to the procedure, which lands and pushes before it tags. The reasoning is in the tool's docstring, not only here, because the next person to notice the gap will be reading the tool.

A citation is a measurement, again, and this time it was caught inside the same increment. The docstring's gap paragraph first read "steps 4 and 5 land and push before step 6 creates the tag" — citing a numbering that only exists in a diff still sitting with the planner. Rewritten number-free. The same pass kept docs/RELEASING.md's steps 6 and 7 unrenumbered for the same reason: five live citations name "docs/RELEASING.md step 8" — in CHANGELOG.md, docs/STATUS.md, docs/ROADMAP.md, .github/workflows/release.yml and tests/test_check_script.py — and pushing that step to 9 would leave every one off by one, silently, in the exact class this repository has now recorded five passes of.

Measuring the review loop — every arithmetic claim in the report had an error (20260826 06:50)

A session measured what this project's adversarial review costs, from the agent transcripts on disk, and proposed eight ways to make it cheaper. An adversarial fan-out over those proposals returned 52 findings. Six of the six arithmetic claims that were re-derived turned out to be wrong, in a report whose whole purpose was to be quantitative. The mechanism it described survived; almost none of its numbers did. What follows is why each one broke, because the causes are not specific to this measurement.

HIGH — a number in a docstring is a claim with no gate on it, and the fix that invalidates it will not say so. tools/review_pass_gate.py shipped citing "613 review agents, 87% leave some artifact, 58% of 6,015 targets are relative paths". During that same increment a filter was added rejecting shell fragments a redirect-scan had reported as files — 0, c, 15,} — because a real run showed them. Adding it changed the denominator of the measurement quoted three paragraphs further down the same file, and nothing re-ran it: 49.2% of those 6,015 "targets" were never files. The true figures are 41% leaving any artifact and 59% leaving none. ./check.sh, pyright, ruff and sixteen tests were green throughout. Lesson: prose in a source file is unversioned evidence. When a change alters what a measurement counts, the measurement is stale even though the diff never touches it — and no gate in this repo can see that.

HIGH — a keyword classifier over agent briefs counts the repository's own instructions. The population was built by matching review over each subagent's task. CLAUDE.md tells every implementer to "adversarially review" its work, so coder agents matched and were counted as reviewers — and coder agents write far more files, and cost far more per turn (median ~173,500 tokens/turn against a reviewer's ~61,700), so they distorted every derived ratio. The replacement classifies on role and was hand-audited over a 30-run random sample: 14 of 14 correct, recall imperfect, so it is a lower bound rather than a point estimate. Lesson: when the corpus is agent transcripts, the vocabulary of the task and the vocabulary of the instructions are the same vocabulary. State a classifier's measured error rate, or do not state a population.

HIGH — the currency was never declared, and the two currencies disagree by 25x. "87% of review spend is re-transmission, 13% is the findings" is a weighted ratio. In raw tokens — the unit every other figure in the same report used — it is 99.5% re-transmission and 0.5% output. Both are true; quoting one beside totals stated in the other puts two denominators in one paragraph. Lesson: a ratio over tokens is meaningless until the weighting is written next to it.

MEDIUM — a pooled ratio is not a typical one. "101,256 tokens per turn" was Σtokens ÷ Σturns across the corpus. Corrected and de-contaminated it is 91,617 pooled, against a per-agent median of 61,688 — the pooled figure sits near the 90th percentile of individual agents, so it described almost none of them. Lesson: pooling answers "what did the fleet cost"; sizing a per-agent budget needs the per-agent distribution, and the two differ by 50% here.

MEDIUM — six merges from one afternoon are not "recent increments". The report argued that review was disproportionate to the work, citing the last four merges at 7, 37, 62 and 232 changed lines. Those were one burst of documentation and fragment merges. Re-sampled, the surrounding increments run 70 to 1,211 lines. An entire proposal — scale the review down for small increments — rested on that sample and does not survive it. Lesson: git log --merges -n returns the most recent n, not a sample of the distribution, and a project that lands in bursts makes those two very different.

And the finding that made the rest recoverable: the fan-out that raised all of this lost every one of its six refuters to a usage limit and returned {"confirmed": []}. Written the way the tool's own documentation recommends — collecting results and filtering out the empty ones — that is indistinguishable from an adversary that examined the work and approved it. The findings survived only because the harness counted launched agents against returned ones and refused. Each was then confirmed by re-deriving it rather than by reading it again. Lesson: an adversarial pass that returns nothing is making a claim about itself first and about the work second, and re-reading a number never falsifies it — recomputing it does.

And the finding the report should have led with: the retrospective loop IS the spend, and its yield halves while its price does not. The report's own share figures were computed from a keyword classifier and were wrong twice. Recomputed the plain way — list every subagent run's task title with its token total, then let the clusters fall out — 899 runs and 5.13 billion raw tokens divide like this: adversarial review of an increment, plus the refuters serving it, is 44.0%; building and fixing increments is 18.1%; plan work 7.8%; decision analysis 5.9%; sweeps 5.7%; corpus and eval authoring 1.5%. A further 16.7% resisted clustering and is named here rather than distributed, so 44% is a floor. Review is 2.4x the next largest category, and the nine single most expensive runs in the project are all build work, which is the other thing the title listing makes obvious and no proposal in the report addressed.

The decay inside that loop is recorded in the repo's own agent briefs, because each pass is told what came before: passes 1–4 over one increment found 30, 22, 13 and 6 issues. Median cost per pass, over the runs that state their number: 7.5M raw tokens at pass 2, 12.5M at pass 3, 12.9M at pass 5 — flat to rising. Yield halves, price does not, so the marginal cost of a finding roughly doubles every pass. The caveat that keeps this honest: only 43 runs state which pass they are, and later passes announce themselves ("the FIFTH adversarial review pass") while a first pass usually just describes the task — so the split of spend by pass number is biased and is not quoted. The decay curve is quoted verbatim from the briefs.

Lesson: the loop's cost is not the depth of any one pass, it is that every pass pays full price for context the previous ones already established. Each pass is a fresh agent, told the history but handed none of the evidence, so it re-derives the map before it can look for anything new — at a measured ~102,000 tokens per turn, where 99.5% of those tokens are re-transmitted context and 0.5% are the findings. That is the shape to attack: not fewer passes and not cheaper models, both of which trade away what later passes are good at, but a later pass that starts from what the earlier ones established instead of from zero.

And the method lesson, which is the one that generalises: the clusters above came from listing every task title against its token total and reading the list. Three successive regex classifiers over agent prompts had each produced a different, confidently wrong answer. Sort the units of work by cost and read the labels before writing a classifier over them.

D-35 layer 2 — the half a gate cannot see, and a gate I ran wrong (20260826 06:59)

HIGH — I ran uv run ruff check . | tail -1 between edits, and it reported tail's exit status. A failing checker read as a pass, and it hid a real E501 for two edits. This is not a new lesson anywhere: check.sh exists because of exactly this, says so in its own header comment, and CLAUDE.md carries it as a standing rule — a gate is only a gate when its exit status is what the next command reads. Nothing false was ever claimed, because ./check.sh runs its gates bare and is what ran before each commit. The generalisable part is where the defect lived: not in the project's gates, which are correct, but in the ad-hoc command I typed to check my work between them. A rule written about check.sh was read as being about check.sh. Piping any checker into head, tail, grep or tee to read its output is the same mistake wherever it happens, and the safe habit is cmd > log 2>&1; echo $? — the exit status first, the output after.

HIGH — the property worth gating was the one no existing check could ever have failed on. D-35's own reasoning is the durable part: an index-based rule can require the marker when a version is absent from PyPI, and can never require its removal once the version is present. Removal is green by construction — in the header gate (its SHAPE stops at the closing **), in the release-order gate (no sequence reads line 3's tail), and in the index rule itself. When choosing what to build, "which direction of this property is currently unfalsifiable?" locates the gap faster than "what is broken?" — the broken direction usually already has a check; the unfalsifiable one has never had a chance to fail.

MEDIUM — the mutant that proves it is the one the real tree cannot show. PUBLISHED_ROW names which of release_order_gate.py's two docs/STATUS.md sequences carries R. Repoint it at the other one — the Published on PyPI prose, forty lines away — and: the real tree stays green, because both sequences have the same newest entry today; the test asserting the sequence resolves stays green, because the prose sequence exists; and every other test stays green. Only a fixture whose prose is absent kills it. Reading the wrong one of those two is a recorded failure in this repository — it is how the row drifted four releases while the gate reported every one of them present, in the sequence next door. A mutant whose kill requires a fixture is exactly the mutant worth committing, and the reasoning beside it is the part that is not re-derivable.

MEDIUM — the gate dictated remedy text it could not hold true. Its failure message told the operator to write "landed on main, NOT tagged and NOT on PyPI". NOT tagged is a claim about git that goes false at the tag, while the version is still unpublished and the Published versions row may legitimately still lag — so the gate would have been green over a marker it had itself dictated and which had become half-false. The qualifier now names the index only. The rule: a check may only prescribe text whose truth it can keep checking; anything else it suggests is a claim it has quietly delegated to a human and stopped watching.

LOW, stated before the build rather than discovered after it — layer 2's green is narrower than it looks. docs/RELEASING.md permits the Published versions row to lag a release, because an entry is held back until it is verified from the index. So after a successful publish line3 > R still holds and layer 2 stays green over a marker that has become false. It enforces the marker's removal at the moment the row is updated, not at the moment of publication; the publication window is layer 3's, which is soft and not built. This is written into the tool's docstring, not only here, because the next person to over-read this gate's green will be reading the tool.

One code change falsified six documents, and the owner of the code could not fix any of them (20260826 07:02)

make release-check became a real gate in 674eda6. The moment it landed, six documents were false — and none of them was in the coder's file set.

Document	What it said
docs/README.md	"make release-check runs no gate at all"
docs/ROADMAP.md ×3	the same claim, at :124, :185, :2274
plans/20260731_1202-open-corrections.md	the item itself, 🛑 LIVE
docs/STATUS.md, CLAUDE.md	"before the tag, never after" — now before the push

MEDIUM — two rules this repository holds simultaneously cannot both be satisfied, and nobody had said so. Keep docs in sync with code — in the same change is a standing rule and calls stale docs a severe defect. Documentation has one owner says no other agent edits a document; it proposes. For any coder change that falsifies a document, the same-change rule is unsatisfiable by construction — the person who knows the change cannot write the document, and the person who owns the document is a separate session. There is always a window. Here it was about twenty minutes and nothing published inside it, but that is scheduling luck, not a property of the mechanism.

The tension resolves in favour of ownership, and the cost should be named rather than absorbed. The mitigations that actually worked, both of them cheap: the coder enumerated the falsified documents in the same message that announced the landing — five of the six, each with a line number, before being asked — and the planner ran the sweep as the next increment rather than batching it. Neither is written down anywhere as the procedure. What makes the window survivable is the handover listing the damage, not the ownership rule being relaxed.

LOW — the counted anchor did not bite, and it was worth checking rather than assuming. docs/RELEASING.md warns that Part 5's Open corrections heading carries its own item count in its anchor, so closing an item breaks every in-page link. Checked: the heading is a bare ## Open corrections at ROADMAP.md:2255 and the counts live in prose at :102, :122 and :186. Corrected 20260830: every one of those numbers was right at c111645^ and wrong the instant c111645 landed — the commit that added this fragment shifted ROADMAP.md by a line. :124 was never a count, so the third is :122, not :124 plus the shift. The warning describes a hazard that is not present in the current text — true when written, and now a trap for whoever believes it and works around a problem that is not there.

LOW — five citations were the reason not to renumber, and they were verified rather than relayed. The coder's diff deliberately kept steps 6–8 numbered as they were, because five live citations name "docs/RELEASING.md step 8" — CHANGELOG.md, docs/STATUS.md, docs/ROADMAP.md, .github/workflows/release.yml, tests/test_check_script.py. All five were grepped before the diff was accepted. Renumbering is the invisible half of a documentation edit: nothing goes red, and every citation is off by one.

S19, measured twice independently — and the row that disagreed is the useful one (20260826 07:20)

S19 said pair() emits an inapplicable order for renames that are not cycles. Until today that was an argument from reading pairing.py, and the plan said so in a provenance paragraph. Both halves have now been run: the coder reproduced it end to end, and the planner re-drove pair() from a separately written probe rather than checking the coder's output.

MEDIUM — three rows matched exactly, and the fourth differed for a reason worth keeping. On cycle, non-cycle-two and chain-of-three the two probes emitted identical action sequences. On rename onto a free name one probe reported RefreshMetadata and the other Skip — because one construction changed the untouched document's sidecar hash and the other did not. The row is green either way and the disagreement changes nothing, but recording why two measurements of "the same thing" differ is what makes the three that agreed worth anything. A silent reconciliation would have left a reader unable to tell agreement from coincidence.

MEDIUM — the control was designed to kill the plausible-but-wrong fix, not to demonstrate the bug. The obvious pinning case is the two-file shift. The coder argued for a chain of three instead: its valid order is strictly reverse, so no accident of path-sorting can produce it, and a fix that merely reorders two adjacent actions passes the two-file case while failing this one. That is the difference between a test that proves the bug existed and a test that constrains the fix.

LOW — and it converted a build-order row from under-scoped to scoped. Before the measurement, "make swaps work" was a defensible reading of S16. After it, the classes are distinct and measured: ordering the applicable plans fixes the whole chain class; cycles need a temporary path and a separate mechanism. The plan now says so, and asks that the cycle case be shown still failing in the commit that fixes the chain — otherwise "swaps still crash" becomes "swaps are fixed" in the next summary, which is precisely how S17 was recorded as open after being cured.

The marker was 0-for-2 because the procedure never asked for it (20260826 07:27)

MEDIUM — a convention with a 0-for-2 record was blamed on the people following it, and the procedure had never contained it. docs/STATUS.md's hold marker had been written twice and missed twice, and that record is what justified building a gate (D-35 layer 2). It is a good gate. But docs/RELEASING.md — the document a release operator actually follows, whose sweep table exists precisely to name every place a release goes stale — never mentioned the marker at all. A release cut by following it verbatim produces the false line, every time, correctly. Before building a gate because a convention keeps being missed, check that the convention is written where the person doing the work would read it. Both were worth doing here; only one of them was diagnosed.

MEDIUM — the live marker carried a claim its own gate had deliberately rejected. Line 3 read "landed on main, NOT tagged and NOT on PyPI". The coder had explicitly cut NOT tagged from the gate's suggested text, because it is a claim about git that goes false at git tag while the version is still unpublished — the line would be half-wrong for the whole tag-to-publish interval, with the gate green over it (HOLD requires only ⏸ and a bold span naming the published version). The reasoning had been recorded in the gate's docstring and the document it governs was never brought into line. A decision written down next to the code is not a decision applied to the data.

LOW, and it is the third instance today of one trap. Driving the gate red on purpose, the first measurement was uv run python tools/status_header_gate.py 2>&1 | grep -v warning; echo $? — which reports grep's status, not the gate's, and printed 0 for a run that had just failed. The coder hit the identical shape with | tail -1 an hour earlier and wrote it up as "a rule written about check.sh got read as being about check.sh". The rule is not about a file; it is about whatever command's exit status you are actually reading — and an ad-hoc pipe typed to inspect a result is exactly where it is forgotten, because it does not look like a gate. Re-measured bare: exit 1 without the marker, exit 0 with it.

The cycle's price, found by building the fix and backing it out (20260826 07:33)

MEDIUM — the obvious remedy for a defect had a cost nobody had counted, and counting it required writing the code and throwing it away. S16's cycle class (a ↔ b) looks like it wants a clean refusal: pair() raises before anything is applied instead of emitting a plan that crashes halfway. The coder built exactly that, it worked, and backed it out — it breaks five committed tests, and three of them are guards that exist only while a cycle still produces a plan, including test_a_rename_cycle_that_fails_halfway_never_destroys_a_live_row, which pins the silent-loss shape S2 exists to prevent and can only observe it by watching a plan be applied and fail.

So the cycle's remedy is not "add a temporary path". It is "add a temporary path and replace three of S2's guards". A build-order row reading make cycles work would have under-scoped it exactly as make swaps work under-scoped the chain — the same error, one class along, and it would have been discovered mid-increment by someone with a fix already written.

The judgement worth keeping is the refusal to land it. Refusing a cycle cleanly is a behaviour decision costing S2 coverage, not an implementation detail of an ordering fix, so it did not belong in that increment however well it worked. Backing out working code because it answers a question nobody asked is harder than shipping it, and it is what the never assume what the plans have not decided rule actually costs when it bites.

LOW — and one of the three guards contradicts itself in a way that will block precisely the person who settles the cycle. Its docstring: "The sync's own outcome is deliberately not asserted… what this test pins is that no live document loses its row, which must hold however that defect is settled." Its body: contextlib.suppress(sqlite3.IntegrityError), which pins the exception type. Settle the cycle by raising anything else and the test fails, with a docstring promising it would not. A test's intent and its implementation can disagree, and the docstring is the half that gets read — nothing checks the other one against it.

Every review pass rebuilds the map, and the expensive ones pay most for it (20260826 07:40)

Measuring where adversarial review spends its tokens, and building the tool that carries the answer forward, produced five things worth keeping. Three are about this repository's review loop; two are about measurement itself, and those are the ones that generalise.

The cost is re-derivation, and it scales with the pass rather than away from it. Over the 910 subagent transcripts here, 35.6% of a later review pass's raw tokens go to turns whose only file access was a file an earlier pass over the same increment had already opened — against 3.0% for turns that opened something new. 96% of what the median later pass opens was already opened by an earlier one. And the share rises with how expensive the pass is: 40.1% over the 69 passes costing more than 5M raw tokens. The reason is arithmetic rather than psychology — a file read at turn 6 of a 90-turn pass is re-transmitted by all 84 turns after it, so the earlier and longer the pass, the more a redundant read costs. Lesson: the expensive part of a review pass is not the reviewing. It is standing the map back up, and the passes best placed to find something rare are the ones paying most for ground that was already walked.

What a pass re-derives is not mainly file content, and counting tool calls said otherwise until the labels were read. Reviewers here overwhelmingly run things: their own detached worktree, their own scratch KB, pytest inside it, a throwaway probe script, then git worktree remove. On one increment, seventeen separate passes wrote and ran uv run --frozen pytest tests/test_pairing.py, and seven independently discovered that timeout is not installed on this machine — each paying full price for the discovery. Lesson: the carry that matters is the probe, not the file list. A map of what was read helps a reader; a command that already ran helps a worker.

And the coverage hole nobody was looking for. The tool reports, per increment, the changed files no pass ever opened: 211 of 248 across the corpus (85%; 92% on multi-pass increments). The misses are not noise. src/pinakes/__init__.py — where __version__ lives — was never opened across the 41 passes over 20260823_0718-mutation-batteries. And no review pass in this corpus has ever opened the changelog.d/ or retro.d/ fragment its own increment wrote: the files that carry this project's memory forward are the ones its review procedure never reads. Lesson: "reviewed until clean" is a claim about what the passes looked at, and until something lists the diff beside the reads, nobody knows what that was.

The method lesson, which is the one that generalises: a mutant that survives is a question, and the answer was that my reasoning was backwards. The battery's first run left one row alive — the mutant that flipped path normalisation from last-match to first-match. The test it named asserted a property the fixture never created, so the mutant was unobservable. Answering why it survived found that the shipped rule was the wrong one: last-match reduced tests/demo-kb/docs/x.md to docs/x.md, which is not a file, and the tracked-file screen then discarded it. 102 tracked paths are shaped that way. The written justification for last-match — that a branch directory might contain docs — was simply false: a branch is 20260807_2143-docs-audit-findings, where docs is bounded by hyphens and the /docs/ marker never matches it. Every gate was green throughout.

Two corollaries, both earned in the same hour:

• Then measure the fix rather than asserting it. The natural write-up was "every review pass that read the demo KB vanished from the map". Checked: 28 of 49,000 read targets normalise differently, 27 of them recovered, and the published shares do not move at all. Real, silent, and small. The defect class deserved the fix; the sentence deserved the smaller number.
• A number in a docstring is a claim with no gate on it, so give it a command. --measure re-derives every figure in this tool's own docstring, under the key printed beside it. Two of those figures moved during the build, when the two defects above were fixed — and updating them cost one command instead of one act of remembering. This is the direct answer to the failure recorded in tools/review_pass_gate.py, which shipped three measured claims that a change made in the same increment falsified.

And the fifth, which arrived while writing the fourth: the corpus is live, so a measurement over it needs a stamp exactly as a measurement over a working tree needs a sha. Two peer sessions were writing subagent transcripts into the same directory throughout this work. Running --measure twice, minutes apart, with no code change between them, moved a published figure from 96.1% to 95.8%. Nothing was wrong; the corpus had grown. --measure now prints transcripts_read and newest_transcript with every result, and the module docstring quotes both beside its numbers.

This is the same lesson a peer had reached from the other direction in the same hour — it dictated counted prose to two branches from two trees, neither containing the other's change, so whichever landed second would have made the sentence false. A stamped measurement that no longer matches the tree is dated; an unstamped one is merely wrong, and looks identical. The two of us met this from opposite ends of one morning, which is the strongest evidence either of us has that it is a property of working here and not of either task.

The review step ran before the fragment existed, so the fragment was never reviewed (20260826 11:30)

A peer measured that no adversarial pass in this repository's transcript corpus has ever opened the changelog.d/ or retro.d/ fragment its own increment wrote, and that passes open only 207 of 248 files their increment changed — with src/pinakes/__init__.py, where __version__ lives, opened by none of the 41 passes over the mutation-batteries increment.

MEDIUM — the cause is not inattention, it is the order of the steps. docs/BUILDING.md lists the adversarial review as step 5 and "a changelog.d/ fragment in the same commit as the code" as step 6. The fragment does not exist when the last review pass runs. The procedure guarantees the outcome the measurement found; no reviewer had to forget anything. That is a better finding than "reviewers skip fragments", and it is only visible if you read the two steps as an ordering rather than as a list.

MEDIUM — and it had already cost something, in the increment that was about exactly this. The changelog fragment written in d9fe1a9 carried "wrong for twelve hours" — a duration that had been invented, and repeated across four files, inside an increment whose whole subject was claims asserted without measurement. The review pass did not catch it because the pass read the plans and never opened the fragment. It was found while re-deriving timestamps for an unrelated reason. A fragment is not scratch: tools/fragments.py splices it into CHANGELOG.md and docs/RETROSPECTIVES.md, and docs/ publishes on every push — an unreviewed fragment is an unreviewed published document.

LOW, and it is the caveat that keeps the rule honest. The rule says opened, not reviewed, and the peer insisted on the distinction before it was written down. A transcript can show which files a pass opened; whether it reviewed them is not observable and no tool will ever check it. So the checkable rule is the weak one, and it is stated as a floor rather than a goal. An opened file is not a reviewed file — the measurement is a lower bound on attention, never evidence of it. Writing the strong version would have produced a rule that reads as a guarantee and is not one, which is the failure class this repository names most often.

LOW — and applying the rule immediately caught a false negative in the checking itself. Verifying that the d9fe1a9 fragment really had carried the invented duration, grep -o "wrong for.*hours" returned nothing — on a file that contains the phrase. Markdown had wrapped it across a line break, and grep is line-based. The first measurement said the claim was unfounded; the second, tr '\n' ' ' before matching, said it was true. A grep over prose is a grep over lines, not over sentences — so a phrase spanning a wrap is invisible to it, and the failure is silent and reads exactly like a clean result. That is the shape of every defect in this increment: a check that answers a narrower question than the one asked, and returns green.

MEDIUM — and opening my own fragments found that I had composed three timestamps today, in seven fragments about not composing things. The repository's most-repeated rule is read the clock, never compose it. Every fragment filename obeyed it — they come from date -u "+%Y%m%d_%H%M". Every fragment heading I typed by hand, and three of the nine disagreed with their own filename:

Fragment	filename	heading I wrote	out by
…0727-a-procedure-that-never-asked	07:27	07:26	1 minute
…0733-the-cycles-price	07:33	07:31	2 minutes
…1130-…-never-reviewed (this one)	11:30	08:00	3 h 30 m

The last is the instructive one: I wrote 08:00 from a sense of how long the session had been running. The clock said 11:35. A composed timestamp is not approximately right — it is unrelated, and the error grows with exactly the thing that makes you stop checking.

And nothing catches it. Measured, not assumed. A probe fragment named …_0101-… with the heading stamp 23:59 — twenty-two hours apart — was reported by python3 tools/fragments.py --check as "all well-formed", exit 0. The filename prefix is parsed (fragments.py:124) only to be stripped before reading the slug; the heading's stamp is never compared to it. Both halves of the fragment resolve, so the same shape as every other defect in this increment: a check that answers a narrower question than the one asked, and returns green.

The fix is a gate and it is tools/, so it is proposed rather than written: --check should fail when a retrospective heading's YYYYMMDD HH:MM disagrees with its filename prefix. Both are in the file; nothing external is needed; and it converts the repository's most-repeated convention — which has now missed three times in one morning, in the fragments of the sessions writing about measurement — into something that cannot miss silently.

The remedy for a bad survivor had a false-positive mode, and it was in the advice (20260826 11:46)

tools/batteries/README.md § Reading a SURVIVED row told you to run the mutant against the whole suite to tell a mis-named witness from a real gap. That advice has a false positive, and it fires on exactly the batteries covering tools/.

MEDIUM — the whole suite includes the test that fails on any edit to a battery target. tests/test_batteries.py::test_every_anchor_still_resolves_exactly_once_in_the_file_it_names checks that each battery's old string still resolves in the file it names — and a mutant is precisely an edit to that string. So the suite goes red for a survivor exactly as readily as for a kill, and "something else killed it" — the conclusion the paragraph tells you to draw — is unreachable from the evidence it tells you to gather.

Measured, not argued. Applying tools-fragments.toml's first mutant to tools/fragments.py (return line.rstrip() == "---" → return False) and running the two files separately:

	exit
the row's own kills selector	1	a genuine kill — the behaviour is tested
tests/test_batteries.py	1	1 failed, 10 passed — the anchor check, not a behaviour test

Restored, __pycache__ cleared, both green again: 51 passed. The fix is one flag — uv run pytest --ignore=tests/test_batteries.py — now in the paragraph, with the reason beside it.

LOW, and it is the part worth generalising: tools/mutate.py was never wrong. It runs only the selector a row names, so its report was honest throughout. The defect lived in the prose that told a human what to do next, and prose is not covered by anything. A tool can be correct, its tests green, its battery complete — and the sentence telling you how to read its output can still send you to the wrong diagnosis. Found by a coder following the instruction literally and noticing the kill was the wrong shape, which is the only way this class is ever found.

And the correction improved on the framing it corrected. The first version of this finding — mine — said a whole-suite mutant run "scored a kill that was an artifact", which reads as a defect in the harness. It is not: the harness never made the claim. Locating a defect in the advice rather than the tool is what makes it fixable in one flag instead of a redesign.

A ratio over a growing denominator, and a stamp that should never have been read twice (20260826 11:51)

Two conventions were relying on someone being careful. Both now say what to run instead.

MEDIUM — three figures I put into docs/BUILDING.md were wrong within four hours, and the second reason is the one that matters. The rule cited 207 of 248 changed files opened, 83%, 90% on multi-pass increments. The first reason they moved is ordinary: the measuring tool's scan could not match an extensionless filename or a .lock, so Makefile and uv.lock were reported as opened by nobody on every increment that touched them — a gap section confidently naming what the tool cannot see. Fixed by its author; the figures became 211, 85%, 92%.

The second reason has no fix: the corpus is alive. It is this repository's own transcripts, and the sessions measuring it are writing into it while it is measured. --measure moved a published figure from 96.1% to 95.8% between two runs with no code change at all. A ratio over a growing denominator has a shelf-life measured in minutes. So the paragraph now quotes the command — python3 tools/review_ledger.py <increment> — and no percentage, which is the same conclusion docs/VERIFICATION.md reached the same morning by a different route: two sessions dictating counts from two trees, neither containing the other's change.

The general form, arrived at from opposite ends by two sessions in one morning: a stamped measurement that no longer matches the tree is dated, not false; a restated count is simply wrong. One of us met it by dictating counts from two trees, the other by watching a corpus grow under a measurement. That it was reached twice independently is better evidence that it is a property of working here than either route is on its own.

LOW — and the counterexample keeps the rule honest. The stamp gate proposed for tools/fragments.py compares a heading's timestamp to its own filename. Both values are inside the file, so it is immune to everything above: no corpus, no denominator, nothing to grow. It is the exception that shows what the rule is really about — not never state a number, but never state a number whose subject can move without the document knowing.

MEDIUM — and the convention that gate will enforce did not exist in writing. retro.d/README.md showed a heading with a timestamp and never said where the timestamp came from, which left name-early, head-late looking like a legitimate workflow and made a tolerance look reasonable. It is now written: the heading's stamp is a copy of the filename's prefix — one reading of the clock, written twice. That makes exact equality correct by construction rather than by taste, and it was a coder's question about tolerance that exposed the gap: a gate cannot be specified against a convention nobody wrote down.

Red on the clock, and five measurements of one string that all failed (20260830 09:29)

MEDIUM — a suite went red with no commit, and the design had already decided it should not. prices.toml aged past max_price_age_days (30) on 20260827 and 25 tests began failing. Not one line of code changed. docs/DESIGN.md §5 had chosen a runtime refusal plus a doctor WARN so staleness would not gate a build — and then 25 tests asserted the un-stale path without pinning a clock, so the suite quietly became the gate the design refused. The defect is neither the constant nor the table's age: it is a test that inherits today's date and asserts an outcome that depends on it.

And the precedent cited against wall-clock checks was this file. tools/status_header_gate.py:52 declines a staleness check because "a wall-clock staleness check fails on a quiet weekend with no code change" — naming prices.toml as the reason not to add one. prices.toml then did exactly that, after a quiet four days.

MEDIUM — two adversarial reviews died the same way within hours, and both reported a clean bill. 52 agents, 26 dead; 19 agents, 17 dead, the second returning a literal {"raised":0,"confirmed":[],"refuted":[]}. Two independent instances in one day is the contract, not bad luck — a dead agent becomes null and .filter(Boolean) erases the evidence. Both were recoverable from agent transcripts on disk, which makes it a procedure rather than a warning. tools/review_ledger.py exits 1 on an unfinished pass and would have caught both before either result was read. A third hole was nobody's session limit: the workflow capped a lens at .slice(0, 12) when it raised 16, and logged nothing. Three of the four dropped were real.

HIGH, and it is about the author rather than the tools: five measurements of one string failed before the truth came out. The claim was that a fragment "written in d9fe1a9" carried the phrase "wrong for twelve hours". Checking it:

attempt	result	why it lied
grep for the phrase	no match	the phrase was wrapped across a newline; grep is line-based
tr '\n' ' ' then match	no match	Markdown indentation left multiple spaces between the words
the same check in a shell for loop	0 for every sha	a quoting artifact; the file plainly contained it
reading 29856b9	"confirmed true"	the sentence names d9fe1a9, a different commit
uv run … \| grep …; echo $? on a gate	0 on a failing run	$? is grep's status

The claim was false and three independent agents said so while the author confirmed it twice. Four of those five failures are the same shape: a check that answers a narrower question than the one asked, and returns something indistinguishable from a clean result. A shell one-liner composed in the moment is not an instrument — and the author is the least reliable verifier of the author's own text, which is the case for spending agents on prose nobody else has read.

LOW — and the handover rule caught itself. All of the above lived only in RESUME.md, which docs/BUILDING.md calls "a convenience, never a carrier": excluded from git, invisible to every other checkout, "cannot discover that one exists". The rule is the planner's own and the planner broke it — by putting a repository-wide blocker somewhere no other session could ever find. That is what this commit fixes.

Unblocking the clock-red suite: the seam was the table, not the clock (20260830 14:31)

MEDIUM — a true observation pointed at a fix that would have reached none of the failures. The impurity was real and was enumerated correctly: four unconditional datetime.now(UTC) reads in callers (cli.py:874, cli.py:1259, extract/claude.py:649, doctor.py:1586), against one existing seam at cli.py:667. The inference — so add seams to the other three — did not follow. tests/test_cli_ask.py drives real main([...]), so there is no now to inject, and seaming all three would have made zero of its 18 failures reachable. Every clause was true; the population the argument was about, the failing tests, had never been examined. The seam that works is the price table, which the repository already did twice (tests/test_deep_loop.py, tests/test_extract_claude.py) without either helper being noticed by the pass that enumerated the clock reads.

MEDIUM — the fix is a fake, so the honest version fakes exactly one field. An autouse fixture that returned a synthetic price table would have taken the committed prices.toml out of the suite's reach entirely. Calling the real load_prices() and replacing only as_of leaves every model price, the FX rate and the parse of the committed file on the real path: a defect in any of them still fails the same tests. The remaining unreachable inch was named before it was accepted — and then measured, not argued: with as_of = "28 July 2026" committed, the suite is still red in 7 places. Nothing was lost from the catch. What was lost was the sentence, since all seven are subprocess gates that report a free-path failure rather than an unparsable timestamp, so test_prices_are_installed_package_data — which already asserts the format of every other field in that file — now parses as_of too.

MEDIUM — a review finding is a hypothesis until a control runs, and two of mine died there. The adversarial pass over this fix raised two gaps: no test pins the CLI refusal on a stale table, and nothing notices a malformed committed as_of. Both were plausible, self-consistent and directly implied by the change. Both were wrong on inspection. test_cli_ask.py::test_a_price_it_cannot_compute_leaves_the_free_command_working already pins both halves of the refusal and its docstring already says a stale table takes that branch; the malformed case is caught seven ways. Written up unchecked, the pass would have added a duplicate test and a docs/VERIFICATION.md row for a promise already rowed. The review's output was one line, and its value was the two things it stopped.

LOW — "the fix is green" is three claims when CI is a three-leg matrix. Fail-fast names whichever leg loses the race, so a run summary reports one leg and hides two. Each leg was run separately rather than inferred: [light] 2234 passed / 126 skipped, [light,pdf] 2352 / 8, [light,pdf,claude] 2356 / 4 — and the six tests that only the extras can reach are exactly the six a [light]-only checkout skips, which is why two sessions counted 19 and 25 all morning without either being wrong.

And the defect's own cause was sitting in the file the fix lands in, written as an assumption. test_a_stale_price_table_warns_and_names_the_setting's docstring opened "the shipped table is current by construction" — one function below the test that failed because it is not. The correction lands in the same change, or the fix leaves the premise that produced the defect in place. A docstring is a claim with a shelf life and nothing checks it.

S16/S19 — A fix that enforces a property globally unpins every test that asserted it locally (20260826 11:47)

HIGH — the finding is about the mutation harness, not about renames, and it is the most portable thing this increment produced. The ordering pass makes "no action writes a path a live row still holds" true of every plan. Two mutants in src-pinakes-pairing.toml died on main to that property being locally violated — a SoftDelete landing after the Adopt it belongs before — so both became invisible the moment the property was guaranteed globally. Neither --check-anchors nor tests/test_batteries.py can see this happen: the anchor still resolves and the kills selector still names a test that exists, which is the entire contract those checks hold. The generalisable rule: when a change makes a property hold by construction, every test that asserted that property by observation stops discriminating — and a battery is exactly a corpus of such tests. Run the battery after a change of that shape; the anchors resolving is not a substitute.

The one that had nothing underneath it was still worth the run. Inverted guard was killed by eleven other tests: the battery was right about the risk and wrong about the witness, which tools/batteries/README.md already names. It was repointed and renamed — its old name described the in-place half, which the ordering pass now repairs, and what still dies is the other branch. A row's name is a claim too, and it rotted without its anchor moving.

HIGH — the one that did have something underneath it was an untested route to breaking ULID permanence. An orphaned sidecar counts as a claim survived on the branch and nothing in 2 212 tests saw it. Reading why found a second reader of claimed_by_id that the ordering had been standing in front of: a file with no sidecar of its own, whose index id turns up claimed elsewhere, is read as "this row's identity has moved" and its path is left to be minted fresh. Let an orphaned .pnk.yaml count and that fires on an untouched document sitting beside it — re-minted under a new id, the original retired, every inbound pnk:// link dead. That is docs/INVARIANTS.md's first invariant, reachable with the user doing nothing wrong, and it had no test. The new one asserts the action kind, not the order — the half a global ordering guarantee cannot repair — and was watched red under the mutant first: Skip becomes Rename.

MEDIUM — the battery README's own remedy has a false-positive mode. It says: before believing a survivor, run the mutant against the whole suite. The whole suite contains tests/test_batteries.py, whose anchor check fails on any edit to a battery target — so a mutant that dies only to that is not killed, and a run reporting 1 failed, 2 221 passed reads as a clean bill. tools/mutate.py is blameless: it runs only the named selector and its report was honest. The defect was in the advice, which makes it a one-flag fix (--ignore=tests/test_batteries.py) rather than a redesign.

MEDIUM — working code was written, tested, and backed out, and that was the right call. Making a rename cycle raise cleanly from pair() is tidier than letting it fail at the first write: nothing is applied, so nothing is half-applied. It broke five committed tests, and three of them are guards that exist only while a cycle still produces a plan — including one pinning the silent-loss shape the previous increment exists to prevent, which can only observe it by watching a plan be applied and fail. Refusing the cycle is a decision about behaviour that costs coverage of a different defect, not an implementation detail of an ordering fix. The price is now recorded where the next person reaching for it will start from it: not "add a temporary path" but "add a temporary path and replace three guards."

LOW, and a lesson about test seams rather than about this code: a stability test that could not reach the thing it tested. The ordering pass returns early when a plan has no constraints — so the test that pinned "a plan needing no constraint comes out unchanged" never entered the topological sort at all. A mutant emitting ready actions in arbitrary order therefore changed the output of every constrained plan and survived that test and the whole suite. A test named for a property is not evidence it reached the code implementing it; the repair was a plan carrying a constraint and untouched actions beside it, so the sort runs and the unconstrained actions are observable.

LOW — ty caught two typing defects pyright strict passed, both in new test code, both real (an attribute read off a union member that does not have it). check.sh calls ty a fast pre-check and never the gate, which remains right — but on this increment it was the only checker that saw them, and that is worth one line in the record rather than a re-litigation of the policy.

A gate green on main and red on the tree main is about to become (20260830 15:04)

MEDIUM — three sessions measured one gate within an hour and got three answers, and every measurement was correct. The planner reported release_order_gate red and asked for a fix. A second session ran it on main at 37689f1, measured green, and — instead of reporting the planner mistaken — said so and named its sha. The planner then re-measured, found it had reported a property of its own uncommitted worktree as a property of main, and corrected itself unprompted. The resolution came from neither: reproducing the failure before writing a line. Copying the three documents to a scratch tree and appending 0.31.0 exactly as the post-publish sweep does produced the planner's two failure lines verbatim.

So the accurate statement is one none of the three had: the gate is green on main and red on the tree main is about to become. Both PyPI sequences declare newest_may_lag, so while 0.30.3 is the tail its absence reads as latency; adding 0.31.0 makes it an interior hole, which lag does not cover. A blocker observable only from a tree that does not exist yet is still a blocker — and it is invisible to exactly the check a careful reviewer runs, which is git rev-parse HEAD followed by the gate. This is the third measurement-over-a-moving-target failure recorded in one day and the only one where the target moved forward: the other two moved backward, a suite read across a release cut and a TZ=UTC git log --date=format: printing a committer-offset time as UTC.

LOW, and the reason the fix needed three tests rather than one: a declared absence and a hole nobody noticed have the same exit status. The green run's (declared absent: 0.30.3 — …) sentence is the entire difference, so one mutant silences that sentence and nothing else, and it is killed by the real-documents test rather than by any of the new ones. The control that matters strips the declaration and requires red on exactly the two PyPI sequences — built on a fixture rather than the real documents on purpose, because before the sweep lands the real tree is green with or without the declaration, so a control there would prove nothing today and something different tomorrow. A test whose meaning changes when an unrelated commit lands is not a control.

And the scope is the whole claim. 0.30.3 is a real release document and only never a published artifact; a blanket exclusion would have made the gate green by erasing that distinction. All three new battery rows attack the scope rather than the mechanism — the mechanism was already covered by 0.11.0's rows — because what is new here is which sequences carry the declaration, and that is data, which no type checker reads.

S18 — the obvious fix would have shipped the defect it claimed to fix (20260830 15:15)

HIGH — the right answer was written down, one layer down, and taking it would have been wrong. sync.py:2190 already handles this case beautifully: it decides from the sidecar's own recorded content_hash rather than from a cache miss, peeks the extraction cache, reuses paid text for free when it is warm, and raises the honest PaidExtractionUnavailableError when it is not. docs/DESIGN.md:1036 names that as the required outcome. So the obvious fix was to stop pairing.py pre-empting it — emit Reembed and let the layer with more information decide. Every sentence of that is true and the change would have introduced a Decision 9 violation.

What stopped it was reading _paid_survivor_in_current_index instead of trusting its name: it selects WHERE id = ? AND state = 'active', so a revived row gets no index-side protection at all. The only remaining guard is sync.py:2190, which reads the sidecar's provenance — and WalkedSidecar carries path, document_path, id, file_hash and nothing else, so pair() cannot distinguish a document the cache could revive from one a free backend would silently downgrade. The fix that removes a false claim would have shipped a silent one, under a commit message about honesty.

That turned an implementation choice into a decision with a price, which is the user's and not mine. Three options with a real trade-off — honest reason only, route to the extractor, or widen WalkedSidecar to carry provenance — were put to the user with the cost of each. They took the first, 20260830: the reason is corrected and nothing else moves. The cost is stated rather than buried, here and in docs/DESIGN.md's row: a user whose extraction cache is warm still pays for text their machine already holds.

The distinction that made it a stop rather than a judgement call is worth keeping. Choosing how to implement what a plan specifies is an implementer's; choosing what it should have specified is not. The plan said the fix "must keep the paid-protection clause and stop conflating retired with changed" — all three options do exactly that, and they differ only in whether a warm cache may spare the user money. That is a question about spending, and no amount of reading the code answers it.

MEDIUM — a census cannot see a reason, so nothing but an assertion on the reason catches it. describe() returns {"PaidExtractionRequired": 1} whether the plan says the content changed or the row was retired, and both are one action either way. The mutant that inverts the precedence — RETIRED if retired else CHANGED — is therefore invisible to every structural check in the module, and it fails in the direction that matters: a user whose file genuinely did change is told nothing about it did. It is the obvious implementation, and it is the one a reader would write.

MEDIUM — a field is not a message. The defect was never in a field; it was in a sentence a user reads. So the claim is pinned twice: a unit test on which reason the plan carries, and a test_sync.py test that drives a real delete-and-restore and asserts what is printed. The unit test alone would have passed against a build whose sync still said "content changed" for both reasons — the seam rule applied to a string rather than to a transport.

LOW — the reproduction that made all of this cheap was six lines and one changed field. The same inputs with state=ACTIVE give Skip and protection; with state=DELETED they give PaidExtractionRequired. A control that differs in exactly one field is worth more than a paragraph of reasoning about what the branch does, and it is what let the fix be scoped in minutes rather than argued.

S16's residue — "containment is one line", and the line was too wide (20260831 21:58)

The review was right about the defect and wrong about the fix, and the two are easy to accept together. Two independent adversarial agents found the same residue on the S16 branch: ordering a rename chain creates a dependency between actions that the executor knows nothing about, so one member failing to index leaves its row in place and the next action collides. Both wrote the same remedy: add sqlite3.IntegrityError to _apply's except tuple. One of them called it "containment is one line", which it is.

It is also one line that swallows chunks(doc_id, ordinal), nodes(kind, key), the links and edges primary keys, and the CHECKs on documents.state, links.origin and nodes.kind. Each of those fires only when Pinakes itself is wrong. _apply records a failure and continues — so a bare catch would file an invariant breach as one document's problem and let the run report success around it, which is the exact silent shape docs/INVARIANTS.md exists to prevent. The finding's severity was HIGH and its proposed fix would have opened a second hole to close the first.

The third option was not in the branch, not in the plan, and not in either review. It came out of asking one question the reviews had not: how wide is IntegrityError here, actually? Reading store.py answered it, and exc.sqlite_errorname — measured on the interpreter rather than remembered from the docs — turned out to separate SQLITE_CONSTRAINT_UNIQUE (2067) from SQLITE_CONSTRAINT_CHECK (275) and SQLITE_CONSTRAINT_PRIMARYKEY (1555). That made a narrow catch cheap, so the decision put to the user was three options rather than the two the branch offered. A review's recommended fix inherits the framing of the review, and a reviewer who has just proved a defect is not the least biased party about its remedy.

The battery found the hole in my own test, which is the part worth keeping. The witness for the narrow behaviour had three negative cases — a duplicate primary key, a CHECK breach, a chunks(doc_id, ordinal) collision — and it passed. It also passed with the sqlite_errorname clause deleted, because all three fail on the column substring alone, so the clause the whole decision rests on was never exercised. The case that separates them is NOT NULL constraint failed: documents.path: the right column under the wrong error code. Three plausible negative cases agreeing with each other is not coverage — they have to disagree about the clause under test, and nothing but writing the mutant asks whether they do.

The exit code came from the adversarial pass, not the build. Catching an exception trades a crash for something quieter by construction, so the question "does pnk sync still exit non-zero?" had to be asked separately. It does — report.ok is not self.failures — but nothing pinned it, and a later edit to ok could have turned a contained failure into a silent success with every test green. It is asserted now.

S3 — a thread id is a slot the OS reuses, not an identity (20260831 22:21)

• A defect in how threads share a handle is invisible to a suite that never starts a thread. tests/test_serve.py had no thread anywhere in it. The bug was not missed by a weak assertion; it was outside the shape of every test in the file. The trigger was a property of the transport, and the transport was the part nobody was testing — serve.py starts no threads, so reading it end to end tells you it is single-threaded, and that reading is what the tests encoded.
• The inherited explanation of when it fires was wrong, and only running it said so. The finding came with a mechanism — a worker retired after ten idle seconds, so the first call after a pause gets a new thread — and it is a correct account of anyio. It is not what fails. Driven through the real mcp.call_tool, an eleven-second pause reproduced nothing: the retired worker's thread id had already been reused by its replacement, and sqlite3 compares ids, so it saw one thread. What did reproduce, on every attempt, was close() — shutdown runs on a thread that opened nothing — and, under six concurrent calls, four of six raising ProgrammingError. The trigger is contention, not idleness. The mechanism was sound, the population was never checked, and the first commit message asserted the pause as though it had been measured; it is corrected in the follow-up rather than left to read as evidence.
• A thread id is a slot the OS reuses, not an identity — and it faked a passing test. The first version of the rebuild test ran two worker threads in succession and passed against the unfixed code. macOS had handed the second thread the id of the first the moment the first was reclaimed; sqlite3 compares ids, saw one thread, and allowed the reuse it exists to refuse. Measured directly: three successive anyio workers reported one identical get_ident() and three different Thread objects. Two consequences, one for the test and one for the fix — the opener must be a thread that is still running when the reader starts, and the per-thread cache is keyed by the thread object, because keying by id both hands a new thread a dead one's handle and makes a dead entry indistinguishable from a live one, which is the entry reaping exists to find.
• The control is what turned this from a plausible fix into a pinned one. Reverting the fix failed three of the four tests in their bodies; the fourth — the handle-count bound — is red under the old design only because the attribute does not exist, so it was checked separately by deleting _reap_dead_threads() alone (assert 12 <= 2). Without running both controls the weak test above would have shipped as a certificate, and its docstring now says which control it answers to.
• The seam is named in the test that has one. Driving real threading.Threads proves the handler is per-thread; it does not prove the transport ever uses a second thread. That half is the whole premise of the fix, so it is asserted against mcp directly rather than assumed — a probe tool that records the thread it ran on. If the library stops offloading sync tools, that test goes red and says so, and nothing else in the suite would have noticed.
• The measurement that corrected it also found a second defect nobody had named. Counting the connections actually opened showed the old connection() testing and assigning one shared slot with no lock: threads arriving together each opened one, the last assignment won, and the losers stayed open, unreferenced and unclosable. That race is why some concurrent calls answered — a thread that won it was using a connection it had opened itself — so the leak was also the thing hiding the failure. Neither would have been found by reasoning about the fix; both came from instrumenting the run and asking how many, rather than whether.

The release step written to catch a stale price, skipped by the release that wrote it (20260831 22:53)

• Five hours separated the rule from the violation, and both are in the same day's history. a2cc944 (20260830 10:57) added docs/RELEASING.md § Before you start step 3, including the warning that "re-stamping as_of without re-verifying the numbers is not a refresh — it is falsifying a measurement" and a table whose usd_per_eur row says "this is the field that actually moves". c7b0bd9 (20260830 15:53) cut 0.31.0 and changed exactly one line of prices.toml: as_of. The rate the table singles out was left at its 20260728 seed. A step's first run is where it is weakest, because the person running it is the person who just wrote it and is checking their own text against their own memory of it.
• The model prices are the field that invites the skip. Step 3's other row says of them "unchanged is the normal outcome", and they were unchanged — claude-opus-5 is still 5.00/25.00, re-confirmed again today. Verifying the field that will not have moved feels like performing the step, and it is the half that carries no information. The order in the table now reads as a trap: the reassuring row is first.
• The staleness guard was healthy the whole time, and guarded the wrong thing. max_price_age_days and pnk doctor's WARN both key on as_of, so a re-stamped table is maximally fresh by every instrument the project owns — which is precisely how a wrong number survives. An instrument that reads the claim cannot check the claim. The only detector is going back to the source, which is why step 3 asks for a named one; an unnamed source is not re-checkable by the next release, so prices.toml now names api.frankfurter.dev in the file itself rather than in a commit message nobody will re-read.
• Two independent sources, because being wrong here would have looked exactly like being right. The ECB reference rate for 20260831 is 1.1596; a market quote for the same day gives 1.1617. They agree to 0.2%, which is the difference between a daily fix and an intraday spot, and either would have refuted 1.08. One source would have been a number I could not distinguish from a typo.
• The error's direction is the reason it was survivable, not a reason it was harmless. 1.08 is below the true rate, so cost_usd / usd_per_eur over-stated every euro figure by 7.4% and every EUR cap bit early. Nothing overspent. But the ledger records cost_usd and the rate on each line precisely so euros can be re-derived later, and lines written since 20260728 carry a rate that was never true — the re-derivation is faithful to the record and the record is wrong.
• Nothing in the suite was watching the number, and the thing that went red was an accident. Checked exhaustively rather than by sampling — every EUR literal under tests/ at 3cf9858, seven candidates, each one opened. Six are immune, and all six for the same reason: they pin the rate where the test controls the input (a local constructor default, a module RATE, a per-call rate=, a fixture deliberately at 1.00, a value put in and read straight back). The seventh, test_cli_ask.py:374, reaches the shipped rate only because conftest.py's prices_never_age replaces as_of alone — and it exists to discriminate €0.21 from €0.26, not to watch FX. test_budget_core.py asserts the committed rate is a string that parses as a Decimal, never its value; check.sh's prices gate checks only that as_of parses, and says at length why it is not a staleness check. So the shipped rate's sole guard was a literal nobody wrote for the purpose — and it fires when the rate moves, never when it is stale, which is the wrong half of the property. A month of 1.08 produced no signal from anything.
• Which makes "fix the coupling" a trap. The obvious follow-up — decouple that literal from the shipped table — deletes the only alarm the rate has ever had, and would read as tidying. The fix is two parts, not one: pin the rate at that call site, and separately assert the committed rate is plausible. A bound is wrong in one place and right in the other, which is why one change cannot do both: a bound at the call site would admit 0.26 and destroy the discrimination the test is for, while a bound on the rate is exactly right, because the property there is "somebody refreshed this" rather than "it equals X". And the honest limit, stated rather than sold: a bound cannot catch stale-but-plausible, and 1.08 was plausible for the whole month it was wrong. The mechanism is step 3; the bound is a backstop against a typo or a seed value left in place.
• The find came from executing the procedure, not from auditing it. Step 3 was read tonight in order to run it, and the defect surfaced in the first thirty seconds of doing so. Two days of document audits over the same tree — including one that examined this plan's own file — had not looked at the value.

A population that could not have contained the thing I was looking for (20260831 22:59)

• I reported a zero from a population that structurally could not contain a non-zero. The claim was that claude-fable-5 — which bills at exactly 2.00× claude-opus-5 — had never been used by a fan-out: "subagent runs: 0; workflow agents: 0 of 868". It reached a planner, and it is on origin/main in plans/20260831_2242-agent-spend-two-findings.md as "an unguarded path, not a realised cost." It is false. Measured with the committed script: 316 fable-5 requests in workflow agents and 235 in subagents, all projects — 551 of 866, or 63.6%, across 26 distinct transcripts, billing on three days between 20260709 and 20260821. The reason I saw zero is that I was counting a main-loop-only population: subagent transcripts live under <session>/subagents/ and were never in the file list. The query was sound, the domain was never named, and the answer was exactly the shape that made it look confirmed.
• The same session had just written that rule down for someone else. The document this error landed in exists to record that a claim resting on a selected population must state the selector. I supplied the zeros for it. A rule you are enforcing on a peer is not a rule you have applied to yourself, and the direction of the failure — a zero, which reads as nothing to worry about — is the direction that gets no scrutiny.
• Committing the script is what found it. The figures were re-derived by the tool rather than recalled, and the fan-out numbers appeared the moment include_subagents was a parameter someone had to pass rather than an assumption nobody had written down. Two other reported figures did not survive the same treatment: the cache-re-write counts (135 events, 43,946,201 excess units) reproduce at no cutoff time under any population tried — the nearest is Pinakes main-loop, which now gives 149 and 45.1M. Those never reached a document; the fable zeros did.
• Before trusting a cross-file total, test the overlap. Main-loop and subagent transcripts share 0 request ids of 20,156 and 23,685, so the combined totals are sound rather than double-counted. That check cost one query and was the difference between a reportable number and a plausible one.
• The fix is a default, not a caveat. The tool as first landed hardcoded its file list per subcommand, so the main-loop-only dollar share a document needs to cite was not reachable by any flag — the committed instrument could not reproduce the number the committed document quotes, which is the whole failure it was built to end. --scope {all,main} now selects it, defaulting to all, and every subcommand prints the population it read. The direction of the default is the lesson: the restricted population is the one that must be asked for by name, because a restricted list does not announce itself — it just returns a zero.
• Then I did it again, in the sentence correcting it. The retraction said the fan-out requests sat "across 26 transcripts dated 20260803 and 20260821". The 26 is right — its selector is all projects, subagent and workflow transcripts only, matched on message.model. The dates are wrong. I printed the first six rows of a 26-row list and read the span off the part I could see; the real span is three days, 20260709 · 20260803 · 20260821. A truncated listing is a selected population wearing different clothes, and it went into a commit message.
• A count is not a measurement until its selector is stated, and the selector fails in both directions. A peer scanning the same corpus got 59 files where I got 33. Neither was careless: 59 is every file where the string claude-fable-5 appears; 33 is every file with a line whose message.model is claude-fable-5, which is exactly the set that billed anything. The 26 extra files are ones that merely discuss it — including the sessions doing the measuring. Measuring a model by grepping for its name counts your own analysis as data, and the more carefully you write about it the larger the number gets.
• A split that cannot be reproduced does not get published, and the fix is a flag rather than a footnote. 316 workflow-agent and 235 subagent requests were true and unpublishable: --scope offered main and all and nothing between, so quoting the split would have meant quoting a number the committed instrument could not produce — the defect being corrected, one layer down. main, subagent and workflow now partition the corpus, asserted by a test rather than by arithmetic in a message: on one snapshot, 20,498 + 6,594 + 17,462 = 44,554 = all, file sets disjoint and their union exact. Run across four invocations instead of one, the same sum is off by two — the corpus grows while you measure it, which is why a figure carries the hour it was taken.
• The same failure has a time axis, and the tool had no guard on it. A peer read a workflow's journal while the workflow was still running, saw 8 started against 7 result, and reported the difference as a lost agent. It was the judge, mid-run; the workflow completed 9 of 9 minutes later. The reading was correct and the question when was this read had not been asked — the same defect as out of what population, one axis over. My own analysis had controlled for it only by observation ("the most recent orphaned journal is 20260826, so none is in flight"), which is a fact about that afternoon and not a property of the instrument. workflows now excludes runs whose journal moved in the last --settle-minutes (default 60) and prints how many it excluded, because a silent exclusion is another unstated population. Journal rows carry no timestamp of their own — agentId, key, result, type — so the guard is the file's mtime, and a journal whose age cannot be read counts as settled: the guard drops what it can prove is recent, never what it merely cannot date. With the guard the corpus reads 55 runs / 868 agents / 157 no-terminal-row; without it, 57 / 881 / 159.
• An aggregate is an instrument too, and it has a selector of its own. The span above was wrong twice: first read off a truncated listing, then "corrected" using each transcript's file mtime, which gave four days instead of three. A peer caught it, and checked the instrument the obvious way — against the union, where mtime and the billing timestamps differed on exactly one file in 26, an error rate small enough to write down and move on. Checked per file it is six of 33, four of them by 46 to 48 days: sessions resumed or recompacted weeks after they stopped billing. Five of the six were invisible because they landed on days the union already held. So a union does not merely lose per-file error, it reports the residue as the error rate — and validating an instrument through an aggregate understates it by whatever the aggregate happens to swallow. That is the population failure one level down, found in the paragraph written to correct the population failure. This lesson is deliberately recorded twice — here, and in plans/20260830_0927-main-is-red-and-a-review-that-half-ran.md § An eighth instance, which registers the incident while this records the principle. The plan closes; this does not. Correct either and you must correct both. The precedent is § Reconciling the queues — two registers of one fact, and an owner with nowhere to queue above: plans/20260825_1252-plans-sweep-findings.md kept two registers of the same facts, a later pass updated the prose and not the table, and twelve of its 27 rows stopped describing the tree with no gate able to see it. A second copy is a decision to maintain two things, not a free backup.
• Four wrong spans in one night, none of them arithmetic, and every one caught by a re-run. The fable zeros (a borrowed denominator), a peer's 59 transcripts (a string match that counted the measuring sessions), another's 14-of-100 (a denominator from a set the rule is undefined on), and mtime-for-billing-day. Each was a correct measurement of something nobody had asked about. None was caught by re-reading a document; three were caught by someone re-running the query differently and one by re-running it on a corpus that had settled. That is the case for the instrument being committed rather than its output being quoted — a number in a document cannot be interrogated, and the disagreement between two people running two selectors is the only thing that reliably surfaced the selector at all.

I corrected a document I had not read (20260831 23:26)

• The first version of this fragment was the defect it now records. It opened by quoting "the queued row" as calling this "a probe silently measuring a question set a rebuild overwrote", and spent its lead bullet correcting that framing as a size too big. No plan has ever contained that sentence. git log --all -S"silently measuring" returns exactly one commit — the one that added the fragment. The row (plans/20260825_1240-run-pinakes-sweep.md:579) says the probe hardcodes its questions path "with no way past it", which is precisely the framing I claimed it lacked. The runbook (plans/20260803_2239-corpus-probe-run.md:34) does say "silently" — about the file being clobbered by the next build, where the silent party is build_rfc_corpus.py, which overwrites unconditionally and tells no one. My sentence compressed two documents and moved the adverb from the writer onto the reader. The original text is in 1d5e7ac and stays there; a retraction whose original has been deleted is a claim nobody can check.
• It came from my own session's handoff table. The previous coder session's handover, pasted in at my start, summarised the row as "stops a probe silently measuring a question set a rebuild overwrote" — one cell in a three-row table. I read a paraphrase as a quotation, corrected a document for it, and put that correction into a retrospective, a commit message and a message to the planner before opening plans/. A handoff table is a lossy summary of a document, and the session receiving it cannot tell a paraphrase in one from a citation. The cheap defence is a file:line in the handoff instead of a summary; the reliable one is opening the file.
• The ownership rule is what kept it out of plans/. Documents here have one owner, so the correction could only be proposed. It landed in the two places an implementer writes for itself — a fragment and a commit message — and nowhere else. That is the mechanism working, and it is worth saying because the rule usually reads as friction.
• What was actually true, verified rather than asserted: the probe has recorded the golden set's resolved path, sha256, question count and multi-hop count in both output formats since a6a931b (20260804 04:13 UTC), three days before the runbook warning and 27 days before the row. So nothing was ever misreported, and the row's own scoping — no route to re-measure a replaced set — was right all along. Three lines, no guard.
• Two reviewers found it; neither found the half that mattered. Both raised it as a fabricated quotation, which it is. Going to the row myself, instead of accepting a confirmed verdict, is what turned up that the correction was also substantively wrong — there was no oversized framing to walk back. An adversarial pass that agrees with you about what is wrong can still be a level short on why, and a confirmed finding is a place to start reading, not a conclusion.
• The same shape twice in ninety minutes, on a denominator this time. Arguing against gating a fragment's heading stamp, I measured 14 of 100 retro fragments whose stamp is not a copy of the filename's prefix. The rule is undefined on the sixteen fragments that predate the naming convention, so the population is 85, not 101; the numerator was right and reproduced independently, and only the denominator was borrowed. A ratio is a claim about its denominator. Recounted on the population the rule actually covers, 18 of 85 were wrong at the moment they were committed — which is an argument for the gate I had just argued against.
• And that recount had an unstated selector of its own, which is the same failure one level in. 85 counts git log --all, so it includes fragments on branches nobody has merged — this one's included. Re-run at a named commit: at d66acb8, 100 fragments ever added, 84 prefixed, 16 not, and 18 of the 84 were wrong at the moment they were committed. That is the planner's published 18 of 84 exactly, reached by a different method. --all and a ref are different populations, and retro.d/ gains files hourly — a census of it that does not name its commit is not a measurement. Caught before it was raised with the planner, by re-running with the selector stated, which is the only step that has ever caught one of these.
• The total is robust and the breakdown is not, which is worth more than either number. Three independent counts agree on 18 and disagree on how it splits: 2/8/8, then 4/3/11, now 4 with no heading at all, 5 with no stamp, 9 with a stamp that is not the filename's. Nothing is wrong in any of them — the boundary between no stamp and wrong stamp is a definition each count drew for itself, and none stated it. A breakdown inherits every ambiguity the total averages out, so the gate is argued from 18, and the split is not published. One byproduct: reading each file at the sha that added it (git show <sha>:<path>) resolves all 84, so the three lost to rename detection in the ruling are an artifact of the method, not of the corpus.

A composed number, again — and a test that was not a pin (20260901 06:31)

Two findings from the adversarial pass over the --questions guard, kept because neither is about --questions.

HIGH — the third block of the guard's own test asserted nothing. It claimed to catch the plausible widening of the guard to the KB's default golden set, and its KB was tmp_path / "kb-without-a-golden-set" with mkdir(). An empty directory has no pinakes.toml, so load() raised ManifestError and execution never reached the default questions path at all. Both assertions — returncode != 2, "no golden set at" not in stderr — were satisfied by that unrelated crash. The reviewer patched the widened guard in and the test passed. It is a real copy of the demo KB now, with only eval/questions.yaml removed, plus a third assertion that the default path was actually reached, and mutant 6 in the battery is the widening it now kills.

The rule this breaks is the house one: a test is a pin only if reverting the fix turns it red. The subtler half is that I never applied it here, because there was no fix to revert — the block asserts a **non-**behaviour, that something is deliberately not guarded. A negative assertion has no fix to back out, so the pin test has to be run forward instead: apply the change the block exists to forbid, and watch it go red. Nothing in the procedure said to do that, and I did not.

MEDIUM — "a nine-frame FileNotFoundError". It is five. Module, main, load_questions, read_text, open, measured on b47eda6. The number went into the commit message, into a comment in shipped code, and into the test's docstring, in the same phrasing three times. Nobody counted it.

That is the second composed number on this branch in one evening, and the fourth of the day across sessions. The first was a quotation attributed to a plan row that the row does not contain; this one is an integer attributed to a traceback nobody ran. The pattern is not carelessness about facts in general — every load-bearing claim on this branch was measured. It is that a number used as texture, to make a sentence concrete, does not feel like a claim while it is being written. retro.d/README.md already names this exact failure for timestamps — "composing it instead is the failure this rule exists to stop" — and the timestamp rule works because it names the instrument: date -u, once, pasted twice. A frame count has no such instrument, so: if a sentence contains a number, either it came from a command in this session's scrollback or it does not go in.

Both were found by review lenses, not by me, and neither would have been found by re-reading: one needed a patched mutant, the other needed grep -c ' File ' on a traceback nobody had generated. That is the same conclusion the day's other four wrong claims reached from four different directions — re-running is what catches these; re-reading is not.

A third pass found the repair worse than the defect, in the way that matters most. Repairing the frame count, I replaced nine with five — and five is the count on Python 3.14. On 3.13 it is six, because pathlib splits read_text across _local.py and _abc.py and 3.14 collapsed it. This project runs both right now: every fresh worktree and CI are 3.14, and the primary checkout is the 3.13 outlier. So the correction was the same defect at higher precision — a number measured once, on one machine, written down as a property of the thing rather than of the instrument. Both numbers are in the comment now, each with its interpreter.

Worse, my own three attempts to measure it produced two invalid numbers before a valid one, and each looked like an answer: the first ran a 3.13 interpreter that could not import pinakes and reported one frame; the second ran the probe from a scratch directory, so its repo-relative default resolved to nothing and it died in load() — six frames, right number, wrong traceback entirely. A count is not a measurement until you have looked at what you counted.

And the same pass refuted a claim I had used to close a decision. I wrote that a golden set which exists and cannot be read could not be pinned here, because chmod(0o000) is refused in this repository and injection cannot cross the subprocess boundary _run_probe uses. The first half is true. The second is false, and it took the reviewer ten lines to show it: _run_probe inherits this process's environment, so PYTHONPATH pointing at a sitecustomize.py runs arbitrary code in the child before the probe's first line. That is the same instrument tests/test_doctor.py:1497 prescribes, for the same stated purpose — what is under test is that an OSError out of the read becomes a message, so raise one — reached through an env var instead of a monkeypatch.setattr.

The shape is worth more than the fix. An impossibility claim is the most expensive kind of unchecked assertion, because it does not merely state something false — it ends an inquiry, and it looks like diligence while doing it. I had written a fifteen-line comment justifying the gap, which made the gap look considered rather than untested; a reviewer who trusted the comment would have stopped where I stopped. The guard now has two arms and the second one is pinned. Before writing that something cannot be tested, spend ten minutes trying to test it — the sentence is a claim like any other, and it is the one nobody re-derives.

Re-running a measurement is not enough when there are two artifacts to run it against (20260901 06:33)

The rule that has caught every one of these is "re-run the measurement". It assumes there is one thing to re-run against. On the night of 20260831–0901 that assumption broke, and the break is worth more than the eight instances it sits at the end of.

Two sessions disputed a count. One had reported tools/agent_spend.py at 2 mentions; the other counted 12. Both numbers were right — different selectors, one stated and one not. In correcting the record, the first session asserted its table had carried the selector on every row. It checked, and reported that it had.

It had checked the wrong artifact. The bash output it generated printed the selector on every row. The message it actually sent reformatted that and carried the selector once. Both artifacts were its own, both were to hand, and they differed in exactly the detail under dispute. The one it opened was the one that agreed with it.

So the mechanism is a substitution, not an omission — and that is what makes it new. The other seven instances that night were a population going missing, a denominator borrowed from next door, a qualifier lost between evidence and conclusion, a ref left unstated. Each is something absent. This one is the right kind of evidence, examined carefully, about the wrong object.

The practical lesson, which is the reason this is written down: "check the artifact rather than your memory of it" is an incomplete instruction, because it does not say which artifact. When several exist, the nearest to hand is the one that flatters. A check has to name its object before it runs, the same way a count has to name its selector.

It happened inside the correction. The claim was made in a message correcting another session's record of unstated selectors, in the sentence inviting that session to check the artifact rather than the memory of it, on the night whose entire subject was this failure family. The timestamps carry that without it needing to be argued.

What actually recovered it, both times, was a peer checking a claim that cut against itself. The second session verified the table, found it reproduced exactly, and struck its own sentence. The first session then re-checked its own correction and found the substitution. Neither was caught by a gate. Of the eight instances that night, exactly one was caught by its own author before it left a worktree — by running a grep before committing rather than after — and it cost one command.

The rule I quoted verbatim had been deleted a minute after I started (20260901 07:13)

HIGH — the file you are told is authoritative is a copy, and the copy has no clock on it.

I settled a disagreement with a peer by quoting the governing rule out of ~/.claude/CLAUDE.md. I quoted it verbatim. I read the surrounding argument. I said which clause carved out my case and why the peer's narrower reading was not what the sentence said. Every discipline this repository has written down for quoting a source, I applied — and I was wrong, because the sentence had been deleted 40 minutes earlier.

My session started	20260901 07:00 UTC
~/.claude/CLAUDE.md last modified	20260901 07:01 UTC (TZ=UTC stat -f "%Sm")
What my system prompt held	the text as of 07:00
What was on disk	its replacement, ruling my case the other way, deliberately

The injected CLAUDE.md in a session's context is a copy, and the copy has no knowable age. The old text read "it does not cover a lone adversarial reviewer judging one finding on its merits" — which reads as a per-finding carve-out, and I built a fan-out of eleven top-tier judges on it. The user had replaced it that minute with a clause naming exactly that case and ruling against it: "a lone reviewer ruling on a single finding on its own merits is a refuter, not the judge — then one Opus pass over the refuters' collected verdicts." The new sentence exists because the old one was ambiguous. I found the ambiguity and resolved it in the direction that had just been closed.

The first version of this entry said the copy "never refreshes". That is false, and the truth is worse. A peer holding the post-edit text in its own injected block corrected it, and both halves are checkable on this machine within the same hour:

	My session	The peer's session
Injected copy holds	Audits and large analyses ALWAYS run on Sonnet 5	env.CLAUDE_CODE_SUBAGENT_MODEL … verified 20260901 07:00 UTC
That string on disk	no match — the heading is gone	present — it is the 07:01 replacement
Vintage	pre-edit	post-edit

Same file, same machine, same hour, two live sessions holding different vintages, refreshed on a trigger neither could observe. Not "old session, old copy" — my start time did not tell me my copy was stale, and the peer's would not have told it that its copy was fresh. Nobody has isolated the trigger and this entry does not guess one.

This is not the relayed-claim failure this repo already has three instances of. Nothing was relayed; I read the primary source. It is one layer under that: the primary source I read was a cached copy of the primary source, of indeterminate age. The rule that falls out is narrow and cheap, and the reason is what the correction changes:

Before quoting CLAUDE.md to settle a dispute, grep it on disk — not because a long session is likely to be stale, but because no session can determine from the inside whether it is.

The peer caught it by grepping the file; I had not, because I believed I was already looking at it.

Two things I did right and one I did not. I stopped the run rather than let the fan-out spawn, edited the stage out, and resumed from the run id — the eleven measure and eleven refute legs were already explicit model: 'sonnet' and cached, so the cost was the eight legs in flight. I did not check the file before asserting from it, and in the same message I raised a sound argument over an unexamined domain against the peer's cost table — whose scope was stated in the message and one grep from confirmable. Invoking the rule is not applying it, and I was the one not applying it while naming it.

The resume geometry, since it is not obvious. Workflow resume caches on (prompt, opts), so editing a late stage leaves earlier legs cached and free. But a stage gated by a parallel() barrier after a pipeline() spawns milliseconds after the last leg returns — there is no window to "let the fan-out finish, then kill before the next stage". With a barrier the choice is stop now and lose what is in flight, or let it complete. Which of pipeline and parallel gates the stage decides whether a zero-waste edit exists at all.

---

Three findings from peers this increment, verified here rather than taken on report. Each was handed over with a wrong cause attached, and in every case the conclusion survived and the explanation did not — which is the argument for re-running rather than re-reading.

• A suspiciously clean result is a reason to check the harness, not to write it down. A peer's 4×4 measurement matrix returned sixteen identical rows. Four programs with different argument parsers do not agree to the character; the uniformity was the tell, not the content. This is a positive test for harness breakage, and it sits beside the mutation rule this repo already has — a run with no kills is a broken harness, not a clean bill — discovered independently, in a different instrument.
• A null result from a selector you did not validate carries no information at all. Two instances, one peer, one hour. First: it grepped for a sentence, got 0, and nearly reported landed work as unlanded — the sentence spans a source line-break, so the contiguous string does not exist even though the text does. Second: it grepped retro.d/ for three phrases it remembered writing, got 0, and was one keystroke from telling the user a landed fragment had been lost — the file is named checking-the-wrong-artifact.md and was in the directory being searched. None of the remembered phrases survived into the committed text. What caught it was an ls in the same command block printing the filename; not the grep, and not a gate.

Distinct from the six refutations ruled against on 20260831: those failed on semantics (the string matched, the claim behind it differed); these fail on encoding (the claim was true, the string could not match). The operational form is testable where "read the surroundings" is not, and the second instance adds an edge to it:

Prove the selector can match something before trusting that it did not — and when searching for a document, list the directory before grepping its contents. A filename is a selector you did not choose, which is exactly what makes it useful.

The asymmetry is what makes this worth the entry — a false negative on did this land? invites rebuilding landed work, and this repository has come within one message of that twice. The substitution half of the same family — running the right measurement against the wrong artifact, and why the nearest to hand is the one that flatters — is already recorded and is not restated here, in the sibling fragment spliced just above this one: Re-running a measurement is not enough when there are two artifacts to run it against (20260901 06:33). This was a code span when it was written, and the link was the correct form even then — it resolves once both fragments are spliced, exactly as docs/RETROSPECTIVES.md:4034 already does. tools/markdown_link_gate.py resolved a #… target against the fragment's own headings, so writing it as a link turned main red at b6be317. Restored here: the gate now resolves a fragment's targets from the document its body is spliced into. - uv run --frozen $cmd cannot work in this shell, and the error message hides why. zsh does not word-split unquoted parameter expansions; bash does. Measured here: c2="python -c"; for w in $c2 yields one word under zsh 5.9 and two under bash. So uv is handed a single argv element named python3 -c and reports Failed to spawn: with the whole string as the command name — which reads as that command failed rather than that is not a command. Build commands as arrays or "$@", never as a string expanded unquoted. Not uv-specific; it is a trap for any loop over commands on this machine.

The near-miss worth recording: that peer's first diagnosis was "python3 is not on the project environment's path, only python is", and RESUME.md carries an instruction to use uv run --frozen python3. Acting on the report would have deleted a correct instruction on the strength of a quoting bug. uv run --frozen python3 -c "import pinakes" prints 0.31.1 in the primary checkout and in a synced worktree alike. A wrong cause with a right conclusion is more dangerous than a wrong conclusion, because the conclusion gets checked and the cause gets copied.

A suspiciously clean result is a reason to check the harness, not to write it down (20260901 07:15)

Sixteen runs agreed to the character, and not one of them had started. Measuring how four programs handle a mistyped --questions, I built the commands in a shell variable and expanded it unquoted: uv run --frozen $cmd --questions …. The result was a table of sixteen rows, every one exit=2 Caused by: No such file or directory (os error 2). It reads as a finding — four tools agreeing on a failure mode — and it is uv never having spawned an interpreter once.

This shell is zsh, and zsh does not word-split unquoted parameter expansions. c2="python -c"; for w in $c2; do … iterates once, over the single word python -c. So uv was handed one argv element named after the whole command line and reported, accurately, that no such program exists. Every run of mine that failed built its command in a string; every run that worked used "$@", already split. Build commands as arrays or "$@" — never as a string expanded unquoted.

What caught it was the uniformity, not the content. Four programs with four argument parsers do not agree to the character; that is not agreement, it is a shared failure upstream of all four. This repository already has the negative form of the rule — a mutation run with no kills is a broken harness, not a clean bill — and this is the same shape in a different instrument. A result too clean to be interesting is evidence about the instrument.

The cost was not the wasted runs; it was the two wrong causes I published in between. First that python3 is absent from the project environment and only python works — false, both import pinakes fine in a synced worktree and in the primary checkout. Then, implicitly, that background execution was to blame — also false, and both were plausible enough that a peer nearly deleted a correct line from RESUME.md on the strength of the first. An invalid measurement produces a confident wrong cause, and the wrong cause travels further than the measurement does, because it is short, it is quotable, and nothing about it looks provisional. Both were caught by someone re-running the thing rather than re-reading my sentence, which is now the fourth time this week.

And the corpus could not have caught the defect this increment fixes. retro.d/ holds only unreleased fragments; the rest are spliced into docs/RETROSPECTIVES.md and deleted at each release. When the gate was built the live directory held 4 files and all 4 passed, so a green run over the corpus proved nothing — the only live counter-example anywhere was a fixture in tests/test_fragments.py, which the new rule promptly turned red. A gate whose population is "whatever is unreleased at this moment" cannot be validated by that population, and the census of the historical fragments is a different denominator that must not be used to predict it.

The same shape again, one instrument along: a kill is not evidence until you check which assertion did the killing. The battery for this gate reported 27 of 27 killed, and one of those kills was false. The mutant was named "the stamp is compared by date alone, so the time may be anything" and was written as clock = "", which makes wanted the string (20260901 :) — a value that matches nothing, so the mutated gate refused every fragment, including correct ones. It is the opposite of the laxity its name claims. It still died, and the report still printed the name beside the test, so the battery carried a coverage claim for an axis nothing tested. The faithful mutant was wanted = f"({day}" — and it stopped being faithful inside this same increment, when the fourth pass anchored the comparison to the end of the line. wanted is re.escaped, so (20260901 then had to be the heading's trailing token, matched nothing, and refused every fragment exactly as its predecessor had, the control among them. Nobody re-measured it, because it was still killed. The row now mutates the anchored comparison itself. A mutant is faithful only against the code it currently runs against; a kill does not carry forward across an edit to a line the mutant does not name.

A wrongly-credited kill is worse than a survivor. A survivor is loud — the report says SURVIVED and somebody looks. A kill is silent, it is what you wanted to see, and nobody rechecks a green row. This repository already holds the negative form of the rule, a mutation run with no kills is a broken harness rather than a clean bill; this is its inverse, and the two together say the same thing about every instrument: the output agreeing with you is not the check.

Two independent things caught it, and neither was a test. Reading the mutation report rather than the code showed the substituted string, and an adversarial reviewer later reproduced it and found the part I had missed — under that mutant the pre-existing control test_the_real_documents_are_clean_which_is_this_checkers_only_control also fails, because the repository's own correctly-stamped fragments are refused. So the kills attribution was not merely mis-named, it was not even exclusive.

And a stamp read correctly off the clock can still be wrong by the time it lands. A peer's closure banner said 07:32 UTC — a true reading when written — and a session-limit stop landed the merge at 11:14, leaving a document naming a time no commit exists at. The clock rule stops you composing a number; it does not stop the world moving between the reading and the write. When a stamp is about when something landed, it can only be written after it lands, or it must name both times and the gap.

I read a runner's summary instead of the gate's own exit status — in the increment whose whole subject is reading the thing itself. ./check.sh > log 2>&1; echo "CHECK_EXIT=$?" >> log, run in the background. The harness reported completed (exit code 0): the wrapper's status, and the wrapper ends in an echo, which always succeeds. I reported the branch green to a peer while CHECK_EXIT=1 sat in my own tail output in the same message. The ; echo "$?" appended specifically to preserve the status is what destroyed it — preserved in the file, discarded in the wrapper. This repository already rules that a gate is only a gate when its exit status is what the next command reads; the new edge is that wrapping a gate in order to record its status mints a second, always-zero status for whoever is watching the wrapper. Read the log's last line, never a runner's summary.

set -e makes a failing gate produce no result for every gate after it, and an absence reads as a pass. check.sh stopped at the link gate, so overlap, nul-scan and template-drift never ran. Anyone reading check.sh failed on the link gate concludes the rest was fine. It was unmeasured — the same class as the wrapper above: a status nobody read, and an absence that looks like a pass.

Two agents mutating one checkout measure each other, and the run reports kills either way. The first draft of this increment's review ran two source-mutating lenses concurrently in one worktree; both apply a mutant to tools/fragments.py and restore it, and a restore by one is a false SURVIVED for the other with nothing in the report able to tell. Rewritten before either finished: the mutators serialized, the other lenses probing a copy through tools/fragments.py --repo <tmpdir>. That is possible only because the tool takes its own root as an argument — a tool parameterised by its repository root is testable by agents that must not touch the tree, which is worth knowing while writing the next one.

Two selectors, no dispute to have — and both numbers wrong anyway. A peer measured five drifted heading stamps shipping; I measured three; we agreed each was right under a different selector, and neither of us had stated one. Re-measured over a population I can name — every retro.d/ path that ever carried a YYYYMMDD_HHMM- prefix, each read at its own adding commit — the triple is six drifted when written, three still wrong in the published document, and three fixed by hand between fragment and splice (fbf17da wrote (20260826 07:31) and the shipped line reads (20260826 07:33); a54b304, whose subject is "applying the new rule to my own fragments found three", makes exactly those three stamp corrections — it changes a fourth file and appends thirty-eight lines of new prose to one of the three above that file's own one-line stamp fix, neither of them a stamp, so it touches four).

And the population needs a measurement point and a selector, because it has both and I wrote neither. git rev-list <sha> --objects, deduplicated, paths matching ^retro\.d/\d{8}_\d{4}-.*\.md$: 111 at 56a970c and still 111 at this branch's tip 6cb80b4 — 107 at the branch point 8540e27, so the number moved inside this branch alone. Three other defensible spellings of the same question, all at 6cb80b4: adding a -- retro.d pathspec gives 102, because a pathspec turns on history simplification and prunes merge sides (adding --full-history back restores 111, which is how that was confirmed rather than assumed); git log --full-history --diff-filter=A gives 109, because a path introduced by a merge was never added by any commit it walks; and plain git log --diff-filter=A gives 100, losing both. Four selectors, spread eleven apart, and the sentence above named none of them until now. The arithmetic was the tell we both walked past: five minus two is not three, and six minus three is. State the selector even when the other party agrees with you: agreement between two unstated populations is a coincidence, not a confirmation.

The general shape of the defect this gate sits beside. This gate reads a relation whose both operands are inside its input — the heading against the filename, one file, nothing else consulted. tools/markdown_link_gate.py reads one operand of a relation whose other operand lives in the destination document. A gate holding half a relation can always be wrong about a form that is correct where it lands, and it is wrong in both directions: green on the branch and broken after splicing, or red on the branch and correct after. One missing operand, two signs. The fix is not a new rule for that gate; it is handing it the operand it never had.

Build the instrument that can call a fan-out wrong before you read what it found (20260901 07:36)

A fan-out re-measured 34 open documentation findings and proposed 2 closures. I could not have judged those 2 from their own text — every closure arrives with a named commit and a passing grep, which is exactly what a wrong closure also looks like. So before reading any of them I measured something the fan-out could not influence: how much each audited file had changed since the audit's own baseline.

File	Commits since c45ffa8	Findings	Closed
docs/VERIFICATION.md	42	2	2
docs/CLI.md · docs/KB-UPDATES.md	4	9 · 4	0
docs/GUIDE.md · docs/DESIGN.md	2 · 1	5 · 6	0
MANIFEST · MEASUREMENT-RUN · both READMEs	0	9	0

Both closures landed in the single most-churned file, and every byte-identical file closed nothing. A prior built before the verdict, from git rather than from the findings, turned two unfalsifiable claims into two predicted ones. Had a closure come back for docs/MANIFEST.md — 0 commits, byte-identical — it would have had to explain how a document nobody edited stopped being wrong, and no per-finding judge could have asked that: each sees one finding, and the rate is the thing no single finding contains.

The cheap general form: when delegated work returns a verdict you cannot check directly, find a second measurement of the same population that the delegates did not produce, and check the two against each other. Here it cost one git log --oneline <range> -- <file> | wc -l per file.

---

Three smaller things, all self-inflicted, all caught by measuring.

• 35 findings, 35 refuter agreements, zero disagreement — and it was fine. My own rule from an hour earlier says a suspiciously clean result is a reason to check the harness. I checked. The refuters had run 263 commands between them and their reasoning showed independent re-reproduction. The uniformity was real agreement, which is what the rule is for: it says check, not disbelieve. A clean result that survives the check is stronger than one nobody questioned.
• My check for "did the refuters do any work?" returned zero, and the zero was mine. I summed the length of each verdict's evidence field; the schema names that field why. Every length was 0 and I was one inference from reporting a rubber-stamp harness. What caught it was printing the key list beside the values in the same command. This is the null-selector rule, hit within the hour by the agent that had just written it down — and the operational half held: printing the keys is the ls before the grep.
• make docs 2>&1 | tail -15 then echo $? reads tail. Twice in one session. The repo's own rule says run a gate bare. Both times the real status happened to be 0, so nothing broke and nothing would have told me if it had. A gate misread as green is indistinguishable from a gate that is green — which is the entire reason the rule exists, and no reason at all to relax it.

---

A number can be wrong in two places for weeks while three routes to it all agree. This audit recorded 39 findings in a heading and a table cell. Counting says 40, and so do the # Medium — 13 / # Low — 27 dividers, and so does 34 open + 6 closed. A correction filed 20260825 had already reasoned it out — "the 34 is right and the 39 is wrong" — and the two cells still said 39 today, because the correction was filed in a third register and nothing read it back. Filing a correction is not making one, and the register that receives a correction is the least likely place anyone will look for it.

The rule that falls out is narrower than "remove counts", and a peer sharpened it while this was being written. Landing beside me, it checked its own new rows in the same file against my decision and found they were exempt — every digit in them belonged to one dated measurement rather than to the tree:

A count of what is in the tree today decays silently; a dated measurement of something that happened once does not, because the tree changing cannot make it false. Take out the first kind. The second kind is why a record can carry numbers at all.

That distinction is the difference between test_eval.py carries 32 rows — false within weeks of being written, twice — and three headings out by 1 minute, 2 minutes and 3 hours 30 minutes on 20260826, which will be true forever. Both are numbers in a document; only one of them is a claim about now.

When a gate and a convention disagree, the shipped artifact is the tiebreaker — and it costs one grep (20260901 11:25)

main went red on a link I wrote. The gate said the anchor did not exist; the gate's own error message said to put it in a code span. Two peers reproduced it independently, diagnosed it identically, and both recommended the same fix. Everything pointed one way: my fragment was wrong, correct it.

It was not wrong. retro.d/README.md:40-48 instructs fragment authors to write exactly that form, and one grep settled which of the two was defective:

$ grep -c '](#[a-z0-9_-]*-20[0-9]\{6\}-[0-9]\{4\})' docs/RETROSPECTIVES.md
2
$ sed -n '4034p' docs/RETROSPECTIVES.md      # links to the heading at :3945
([*measured the launcher, not the work*](#measure_sync_cpupy-measured-the-launcher-not-the-work-20260805-1737))

The form is live in the published document and mkdocs --strict passes over it. So the link was right about where it was going, and the gate is half-built: tools/markdown_link_gate.py models a fragment's disappearance at splice time (:296-305, because --apply deletes it) and not its destination. The fix I applied is still the code span — main was blocking three branches — but it is now recorded as a temporary degradation of a correct link, with a build-order row to remove it, rather than as a correction. Those are different entries in the register, and only one of them gets undone later.

Two things to keep.

A gate is evidence about the tree, not about the convention. When a checker and a documented instruction contradict each other, at least one is a defect and the checker is not automatically the survivor. The tiebreaker is the artifact the convention exists to produce — here, the published document, where the disputed form already worked. That is a grep, not an argument, and it is available before any of the reasoning starts.

Two peers agreeing is not two independent checks when the error message framed them both. Both reproduced the failure faithfully and neither asked whether the form already shipped, because the gate's message had already named the fragment as the thing to fix. Convergence measured agreement with the framing, not with the tree. The question that separated them from it — does this form already work somewhere? — was not a better inference; it was a different population, and nobody had named one. It is the sibling of a null result carries no information until the selector is shown able to fire: a shared premise makes independent agents into one agent.

Related, and the reason this fragment exists at all: the same landing found that four of the eight rows parked in plans/20260825_1240-run-pinakes-sweep.md were already done — a register whose stated purpose is to stop work ageing, ageing at 50%. Every remaining row now carries a command that says whether it is still live, because a row is a claim about the tree on the day it was written and nothing re-checks it.

A register decays at the speed of its neglect, and the rate is measurable (20260901 11:40)

Two registers live in one file. I measured both against the tree on the same morning, with the same method — one agent per open row, plus a spot-check of two verdicts by hand.

Register	Rows checked	Stale	Rate
## Build order — the table an implementer opens to pick up work	7	1	14%
Decided work with an owner and no build order — the parked table	8	4	50%

Same file, same author, same discipline, three and a half times the rot. The variable is not care; it is readership. A stale row in the build order meets a reader who can falsify it, because that table is what somebody opens when they want work. The parked table's stated purpose is that nobody has to read it — and half of it described finished work, two rows of it finished before the table was written.

The one stale build-order row is the more interesting half. Row 3 said the per-thread connection in pnk serve was still to do; e526e29 had landed it nine days earlier, with five tests, one of them asserting the row's own symptom. It was found by a peer reading src/. No gate saw it, and no gate can: a row is a claim about the tree on the day it was written, and nothing re-reads it.

So the two fixes are different, deliberately. The parked table got a liveness command per row — one grep that says whether the row is still live, because the cost of the check is a minute and the cost of skipping it is rebuilding landed work. The build order got a dated measurement per row, with the citation that settles it. That distinction is a peer's and it is worth keeping exactly as they put it: a count of what is in the tree today decays silently; a dated measurement of something that happened once does not, because the tree changing cannot make it false. A row saying "open" rots. A row saying "measured still open, 20260901 11:35 UTC, sync.py:693 catches PinakesError only" cannot — it stays true about that morning, and its staleness becomes visible rather than silent.

The general form: a register nobody reads is not a register, it is a backlog with better punctuation. If a list exists so that work is not forgotten, and the list is never read back, then the list has become the thing that forgets. Either give it a reader or give each row a check that a reader would have run.

I typed the counterevidence myself, then named the wrong mechanism anyway (20260901 11:41)

Both published documents in the primary checkout were truncated to zero and every retro.d/ fragment deleted, README.md included. I measured it, checked no process was still writing, backed up what was untracked, restored from HEAD, and verified. That part was right.

Then I explained it, and the explanation was wrong. I said a peer's in-flight test had written a fixture into the real retro.d/ and run the splice against the real repo root — a test-isolation defect in their code. I wrote it into the build order as a work row assigned to them, and sent it to a second peer who publishes a document that updates in place.

The refutation was already in my own message. I had written, to argue this was not a normal splice: "a real --apply adds content and never touches the README." Both halves are true, and both halves kill the theory I went on to state in the same paragraph. One grep confirms it:

tools/fragments.py:121   fragments_of ... if p.name != "README.md"   -> a consume step cannot delete it
tools/fragments.py:590   (repo / stream.target).write_text(spliced)  -> --apply GROWS the target
tests/test_fragments.py:40   def run(repo: Path, ...) -> `repo` is a required positional; there is
                             no default to forget, so the route I was guarding does not exist

The actual cause was one agent, one block, one failed cd — four newline-separated shell lines in a probe: a cd into a directory that did not exist, followed by rm -f retro.d/*.md and : > on both targets, the pair of them twice in the one run. The cd failed and stopped nothing, so all of it ran in the session's cwd, which was the repository.

That sentence is the second version of this paragraph, and the first was worse than the error it replaced. Told that a second agent had run the same recipe six times three minutes earlier, I ran a census over every transcript, found six more rm -f retro.d/*.md calls, and rewrote the cause as two agents authoring one symptom each. A peer then read the tool results — which neither of us had opened — and the account collapsed: all six of the other agent's runs succeeded inside their probe directory, the first result opening no matches found: retro.d/*.md because that directory was empty. It deleted nothing. The one block whose result reads (eval):cd:1: no such file or directory is the only one that ran anywhere near the repository, and it is the one that did all of the damage.

A command's text is not evidence that it ran where you think it ran. That is the lesson, and it generalises past this recipe in a way that nothing about cd does. My census was one layer short of right: it correctly separated the executions from the two of us grepping for the string during the investigation — eight of the sixteen recorded calls are the investigation — and then treated all eight executions as equivalent, because I was still reading intent. The result is the event. Pair each tool_use with its tool_result on id and read the first line; it costs the same query.

And the argument I did check was one that could not have distinguished the cases. I reasoned that the surviving backup held two-spaces-after-hash.md and no leading-tab.md, so the later block must have run elsewhere — true, and I was right about that block. But the same evidence is equally explained by the damaging block's own second rm -f deleting the file its first half had written, which is what actually happened. I took a correct conclusion from an argument with no discriminating power and counted it as verification.

The failure is not that I guessed. It is that I reasoned from symptoms to a named mechanism in someone else's file without opening the file, while holding — and having already stated — the fact that refuted it. An hour earlier I had written a fragment congratulating myself for running exactly this grep against docs/RETROSPECTIVES.md before deciding which side of a contradiction was defective. I did not run it against fragments.py.

Two things to keep.

A fact you state in passing is not a fact you have used, and the test is mechanical: would the conclusion survive if that sentence were true? Mine would not have. The counterevidence was not missing, or buried, or expensive — it was in my own outgoing sentence, deployed for a smaller purpose, and never turned on the larger claim. When a paragraph contains both "X cannot do this" and "X did this", the paragraph is the alarm; nothing else has to fire. The testable form is a peer's, and it beats "be more careful" precisely because it can be run against a draft.

That peer also drew the distinction I would have missed about my own error: this is not the failure of not looking. I looked, wrote down what I found, and then reached past it for a cause that fit the symptoms. It is the harder one to catch, because the usual remedy — go and read the file — has already been performed. Three of us reasoned from symptom to named mechanism on this incident, and only an agent transcript settled it. The missing instrument was not a gate.

A diagnosis that assigns work to another agent's file is a claim about that file, so open it. Naming a mechanism converts a symptom into a task with an owner, and the owner then has to spend a message refuting it. The bar for "I think this is what happened" and for "row 15, owner: coder" is not the same bar, and I used the first to write the second. The row is removed; the correction cost two peers a round trip each, which is the cheap version of this mistake.

Two of us searched for how a file was deleted; it had never been deleted (20260901 11:53)

A fixture written into the real retro.d/ by a runaway probe was gone by the time anyone looked for it. A peer went after the mechanism properly: grep -r 'git clean' across every *.jsonl under the project's session directory, then grep -r for any rm naming the file. Both returned nothing. They reported it honestly — "an untracked file with no git record and no rm on record is exactly the class of event that leaves no evidence, which is itself the finding" — and I had reached the same dead end from the other side.

The file was in my own scratchpad. I had moved it there during recovery, when I backed up what was untracked, and neither of us thought to look in the place the recovery notes said it went.

The error is not that we missed a directory. It is what we did with a null result. The standing rule here is that a null result carries no information until the selector is shown able to fire — and these selectors had fired: the same greps, over the same files, returned a583's six rm -f retro.d/*.md calls and acea's two. The instrument was proven on the run that produced the zero. So the zero was not an absence of evidence; it was evidence, and what it said was the premise is wrong — nothing deleted the file because the file was never deleted.

That is the half of the rule I had never written down. Once the selector is proven, a null stops being a gap and becomes a finding, and the finding is usually about the question rather than the world. We each kept the question ("what deleted it?") and spent the result on doubting our coverage.

The tell was available and cheap. Both of us had already searched for the deletion; neither of us had searched for the file. One find over the enclosing directories would have ended it in seconds, and find is the query that does not presuppose the event. When a mechanism search comes back empty on a proven instrument, the next query should drop the mechanism, not widen it — ask where the thing is, not what removed it.

Fourth time on this one incident that a symptom was matched to a named mechanism before the premise was checked, and the first where the unchecked premise was mine.

An options table can be honest, complete, and wrong about the question (20260901 12:00)

A survey session put a process decision to the user: freeze the register layer, freeze nothing, or consolidate. Four columns, real pros and cons for each, costs, reversibility, a marked recommendation, and the author volunteered that they benefited from the recommendation being adopted. By this repository's rule for putting a decision to a user, it was correct work — and it was the best-argued option table I have been handed here.

All three options shared a premise none of them stated. The evidence was a ratio: over eight days src/ took 5.5% of inserted lines while plans/ grew 44%. Every option treats that as a measurement of appetite for process — too much, acceptable, or a thing to consolidate. It may instead be a measurement of queue position. Over that same window the code work was decided-but-unstarted and the document work was unblocked, and agents write what is available to write. If that is what happened, freezing the register layer does not cause the unbuilt defects to be built; only building them does, and the ratio corrects itself when the queue drains.

That reading is not a fourth option. It is an argument that the question does not need answering yet, and it cannot appear as a row in a table whose rows are all answers.

What generalises. A pros-and-cons table is a strong instrument for choosing between framings and a weak one for noticing you are inside one — every option inherits the framing, so the framing is the one thing the comparison cannot test. The check that works is cheap and does not require a fourth column: take the evidence and ask what else it would look like this if it were true. Here the evidence was a ratio, and a stalled queue produces the same ratio as an appetite for process.

It is the same failure the rest of today was made of, one level up. A symptom was matched to a named mechanism — a ratio to a cause — and the alternative mechanism was never enumerated. The difference is only that this time the reasoning was laid out in a table with its costs priced, which made it look like the enumeration had already happened.

I added the reading to the file and left the recommendation standing, because the decision is the user's and a third reading is information rather than a ruling. Recording the conflict of interest in the document, not only in the handover message, is part of that: the person choosing should see who benefits from the recommendation without having to read the correspondence.

A watcher outlived the session that armed it, and reported green on a red build (20260901 12:26)

A monitor fired into this session saying CI 9cc9bc1: success. The CI workflow on 9cc9bc1 concluded failure. The event was not wrong about a different sha and not stale — it named the sha under discussion and asserted the opposite of the truth.

It had no owner. Its task belonged to the planner session this one replaced. That session's context was cleared; its monitors were not. They keep running, keep matching, and keep emitting into whoever occupies the project next — with nobody left who knows what filter was armed or why.

That is worse than a stale document, and the reason is about how each is read. A document is dated, and a reader discounts it. An event arrives looking current, and a watcher's whole purpose is to be believed without being re-checked. Every other disputed claim today was settled by running a command; this one arrived pre-packaged as the output of one.

Then two agents got the attribution wrong in opposite directions, and one cat settled it.

	Claim	Actual
Me	"your watcher reported a false green"	Not theirs
The peer	"that was my watcher, mislabelled — a true statement about docs read as CI"	Their two watchers said exactly what they said; the event came from a third session

Their defence of their own instrument was correct and my accusation was not. Their inference that the event must therefore be theirs was also not. Neither of us opened the file before asserting, and the file is three lines long. It is the day's shape for the sixth time: a symptom matched to a named mechanism before the premise was checked.

What the file cannot tell us, stated because the temptation is to close the story. CI <sha>: success does not reveal which defect produced it — a match on the first completed run (the docs workflow) mislabelled, or a read of the CI workflow with the wrong field. Those are different bugs. Without that session's monitor command the mechanism stays open, and the checkable part is enough: the file says success, the workflow failed, and no live session can correct it.

One more, caught before it landed rather than after. Writing the fix for a register that decayed because its measurement was unpinned, I stamped my own replacement measurement 12:26 UTC — a minute that had not happened yet. I caught it by reading the clock before writing the fragment, re-ran the measurement, and took the stamp from the same invocation as the command. The values were unchanged, so nothing shipped wrong; the process was still the exact failure the edit was repairing. Reading the clock and reading it at the moment of the measurement are different disciplines, and only the second one survives being audited.

And the run that failed is worth separating from the failure. upload-artifact timed out against GitHub's own service; every test and gate passed. But eval-cross-machine-compare then skipped, so for the length of attempt 1 cross-machine eval determinism was unverified at that sha — not failed, unverified. Had the upload succeeded and the comparison failed, the summary would have looked identical. A gate that did not run is not a green gate, and a re-run can clear the red X while leaving the same hole.

It did not, here: attempt 2 concluded success with all 15 jobs green and eval-cross-machine-compare ran rather than skipping — checked against the jobs API, because the run's own conclusion is the number that would have hidden the skip in the first place. The distinction still cost nothing and bought the only question worth asking about a re-run, which is not did the red X go away but did the job that was missing execute. And it does not carry forward: both are properties of 9cc9bc1, not of the day, so a later sha re-opens both.

A section number is only meaningful inside its own document (20260901 18:26)

A cross-document reference lost its document and kept its number. docs/README.md described what KB-UPDATES.md still leaves unbuilt as "the rest of §8's shape — pnk adopt". KB-UPDATES.md §8 is Open questions, and it has never contained pnk adopt — not in the current file and not in any commit, which git log -S settles in one command. The §8 that proposes the command is docs/graph/PINAKES_APPROACH.md's, and the reason the number travelled is visible in that file: its release-mapping table has a From column whose cells read §3, §8, R1 R6 — its own sections. Someone read the template release ⚠️ |pnk adopt… | §8 and carried §8 into a sentence about a different note. The bare §N form is what made it portable. A reference that had read PINAKES_APPROACH §8 could not have been misfiled.

Two registers agreed with each other and neither was checked against the tree. The dangling §8 had been repeated between docs/README.md and my own draft rewrite of the KB-UPDATES.md header — I was about to propagate it into a second file, sourced from the first, because the first was a document I trust. What broke the loop was not scepticism about the claim. It was checking whether the string was in the file at all: grep -n 'pnk adopt' docs/KB-UPDATES.md returned only the line I had just written. The selector was proven able to fire — the same pattern across the repo returns eight other hits — so the empty result was a result.

And the header that prompted all this failed by accretion, not by error. Every clause in KB-UPDATES.md's status header was true when it was written; four releases each appended one, and the reader meets them in the order they were added rather than the order they matter. The final clause said --apply remains a proposal; it had shipped two releases before, and §1 of the same file cited "the header above" as agreeing with it. An append-only status line degrades even when nobody writes anything false into it — which is the case for rewriting to the current state rather than layering a correction, and the reason that convention exists.

A gate spoke correctly about an input nobody had edited (20260901 18:40)

MEDIUM — ./check.sh went red on a branch that had touched nothing near the failing gate, and the cause was not in the source tree at all. The repository's standing rule is assume other agents run concurrently, and the failure mode it names is a clean-but-wrong merge. This is one layer below that. Nothing was merged. Nothing was edited. git status was clean throughout. What changed was which tree the worktree's virtualenv resolved pinakes to.

The facts, which are two runs on one tree an hour apart.

Earlier run, nothing else touching the worktree	2420 passed, 4 skipped, no failures
Later run, six review agents working against the same worktree	1 failed, 2419 passed, CHECK_EXIT=1
The failure	test_the_real_package_is_refused_from_the_source_tree expected "resolved to the source tree"
What it got	wheel-import: pinakes resolved to an unpacked tree at /private/tmp/claude-501/… — a scratchpad path
Re-run alone, minutes later	23 passed; the gate invoked directly names this worktree's own src
The venv	two .pth files, the editable one pointing at this worktree's src, by then correct

The inference, which is an inference. Between the two runs the venv's editable pinakes install pointed at a copy of the tree in a scratchpad, and uv run --frozen put it back. Six agents were running against that worktree under an instruction to copy it into their own scratch directory before experimenting. Which one, and by what command, is not established, and no agent is named here — the honest limit of the evidence is that six were running under that instruction.

The repair was visible, and that is the sharper half. uv run --frozen printed Uninstalled 1 package … Installed 1 package above the test output before the tests ran. So the log held both the damage and its cure, in order, and a reader scrolling to the failure would have found a red gate a branch that never went near it. A silent repair leaves a mystery; a visible one leaves something worse — a plausible wrong answer, in the log, above the evidence against it. The line that explains a failure is often printed before the failure, where nobody is looking yet.

The rule. A red ./check.sh is not evidence until you have re-run it with nothing else touching the tree. The same holds for a green one: a run made while agents work in the same worktree is measuring a tree that was not the tree at any single moment. Landing on such a run is landing on a measurement of something that no longer exists.

Why git status cannot catch this class. Every guard this repository has for concurrency reads tracked files — tools/shared_file_overlap.py compares file sets, a monitor watches the primary checkout going dirty, land.py refuses a worktree holding an untracked file. .venv/ is gitignored, so the one piece of state deciding what import pinakes means is invisible to all of them. The seam is not the source; it is everything the source is resolved through.

What it cost and what it nearly cost. Twenty minutes, and one report to a peer that would have read as this branch broke the wheel-import gate. It did not go out that way, because the first move was to re-run the single file rather than to explain the failure — the same discipline as reading a gate's own exit status instead of a runner's summary, applied to a gate whose input rather than whose status had been substituted.

git merge-base is not the branch point after a rebase (20260901 20:03)

Four agents in one review pass raised the same false finding, and every command each of them ran was correct. A comment in a mutation battery said the live retro.d/ directory "was 4 files when this was built and all 4 already passed". Two independent lenses reported it false; two independent refuters, told to refute, each confirmed it false instead. One went further and re-ran the gate at the merge-base to produce a measured count of 12. The sentence was true.

The branch had been rebased. git merge-base HEAD origin/main returned 8540e27, and everyone read that as the branch point. The reflog says otherwise:

…@{8}: branch: Created from main      -> 0aea036
…@{2}: rebase (finish): … onto 8540e27be…

0fd4a86, the commit that wrote the sentence, has author date 2026-09-01T08:19. 8540e27 has author date 19:00 — eleven hours later. The merge-base was a commit that did not exist when the sentence was written, and eight retro fragments landed on main during those eleven hours. At 0aea036, retro.d/ holds exactly four fragments and the gate passes all four.

A rebase moves the merge-base forward and leaves no trace in the commit graph. After it, the merge-base names where the branch was replayed onto, never where its work began. Any claim of the form "when this was written, the tree held X" must be measured at the reflog's branch: Created from, or at the authoring commit's own parent — never at merge-base.

What makes this worth recording is not the wrong number, it is the four-agent agreement. The finder and the refuter are supposed to be independent, and they were: they ran different commands in different orders. They agreed because they shared a premise nobody stated — that merge-base is the branch point — and no amount of independent re-measurement can catch an error in the population you both selected. This repository already carries the rule: a claim resting on a set you selected or an instrument you chose must state the selector. Here the reviewers failed it and the author had not: the sentence names its own selector, "when this was built", and the reviewers substituted a different one.

The cheap defence is to make the selector executable. The sentence now reads with its sha in the neighbouring comment, so the next reader is told which tree to measure rather than left to infer it.

I broke the table I was adding a row to, twice in one day (20260901 20:36)

A row appended before the right anchor is not in the table. I inserted two rows immediately before the paragraph that follows the § 3 build order — which put them after the blank line that ends the table. Markdown then reads them as a second, headerless table. The rows were correct, the cell count was correct, every pipe was escaped, and the result rendered as two tables where the build order is supposed to be one.

What caught it was counting, not reading. awk over the section printed a line number for every table row: …168, 169, 171, 172. The gap at 170 is the whole finding. Reading the file would not have shown it — a blank line between two table rows looks like nothing, and both halves render. The check that works is is the sequence contiguous, which is a property of the sequence and not of any row, and every check I habitually run reads rows.

That is the second table-structure defect here today and the first had the same shape: an unescaped | inside a code span in a measurement row, where GFM splits cells before inline parsing, so a pipe inside backticks still ends a cell. Both are invisible to a reader, invisible to the link gate, invisible to mkdocs build --strict, and fatal to the thing the table is for. Nothing in this repository checks that a table is well-formed — not check.sh, not CI. The counting one-liner is not a gate; it is what I happened to run.

And the file this happened in is the plan about registers decaying. Its own § 0 note records that a register decayed inside the plan about registers decaying, within four hours of landing. This is the same file, six hours later, decaying structurally rather than numerically while I added a row ruling how careful other people should be with headings.

Postscript, same hour, worse. Ruling on that table's row 15 I told the coder to write unreleased, 20260823 · into a battery section. The gate it describes shipped in 0.30.0 — the coder checked, refused the instruction, and gave me the ancestry. I re-derived it rather than accept it: v0.30.0 is the earliest tag containing 6d7c9e3, and v0.29.0 is not, so the selector discriminates. My instruction would have written unreleased over a released property in the one file whose job is recording which release shipped which property.

I had the convention in front of me and read the wrong half of it. tools/batteries/README.md says sections name the release that shipped the property, and only unshipped work says unreleased, YYYYMMDD. The bare date 20260823 is raw material for both spellings, and it reads as unreleased-ish precisely because the unreleased form is the one that carries a date — the version form carries none. That is the coder's observation, and it is the useful part.

The rule I take from it: a non-conformance does not tell you which conforming form it should have been. I inferred the target from the shape of the defect instead of from the fact — and the fact was one git merge-base --is-ancestor away, in a repository whose standing rule is that a peer's claim is not evidence until checked. The peer checked mine.

A clean result from an instrument that never ran (20260902 00:04)

Twice within twenty minutes, two unrelated instruments returned a clean result over a population neither had reached.

The first was this increment's eighth adversarial pass. It returned verdict: CLEAN, findings: []. Its five lenses had all been killed by a session limit before doing any work: agents_done: 0, agents_error: 5. The verdict was not wrong about its input — my own script returns CLEAN when the finding list is empty, and the list was empty. It was wrong about what an empty list meant. Passes 5, 6 and 7 had each found a false sentence, so this CLEAN would have ended the loop. Reviewing the same commit by hand instead found two more false sentences, both written by me in the pass before.

The second was a probe I wrote minutes later, asking how many retro.d/ fragments have ever carried a second ## heading. It read each fragment at the last commit that touched it. For a fragment consumed into docs/RETROSPECTIVES.md at a release, the last commit that touched it is the one that deleted it, where git show <commit>:<path> yields nothing — so it examined 15 of 127 and skipped 112 in silence. It reported 0 of 127. Reading each at the newest commit where the file still existed reports 2 of 127, and those two turned out to be exactly the evidence a peer's open question needed.

And that denominator names no measurement point, so writing it down falsified it. 127 is the count reachable from 2914b09. The commit that recorded 2 of 127 added the 128th path, and the path it added was this file. Pinned: 127 at 2914b09, 128 at 04497a9 and at this branch's tip, 129 at origin/main today, 133 under git rev-list --all --objects — which counts whatever every ref happens to hold and so moves under other people's commits, not just your own. State the sha. The numerator never moved: the same two fragments at every point measured.

The standing rule here covers the step after this one. It says a null result carries no information until the selector is shown able to fire, and that once the selector is proven a null becomes a finding. Both selectors here were fine. grepping for a second heading works; an adversarial lens works. What neither run established is that the instrument ever reached its population — a different question from whether it can fire, and the one that failed.

So the number to demand is the denominator, and it is never the default output. agents_done against agent_count; examined and skipped against the list you started from. I had to add both by hand, and in both cases the honest count is what exposed the result. Until an instrument says how many items it actually looked at, it has not reported a result — because CLEAN, 0 findings and 0 of 127 are precisely what a real clean run looks like, and nothing in either output distinguishes the two.

I measured the live directory and answered about the corpus (20260902 00:07)

A ruling needed to know how often a retro fragment carries a second ## heading. I counted the live directory — 0 of 14 — and told a peer the question was "entirely prospective". The number was right. The sentence was about a different population, and it was wrong: across every fragment that has ever existed there are two, and one of them is the case the ruling turns on.

retro.d/ is a consuming directory. Every release splices its fragments into docs/RETROSPECTIVES.md and deletes them, so the live directory is not a sample of the corpus — it is the handful written since the last release. Counting it and generalising is not a small extrapolation from a large sample; it is a claim about 129 things made by looking at 14. The peer found the two, and had it not, the ruling would have been justified by a population that excluded its own counterexample.

Then the correction repeated the mistake one level down. Asked to check the full history, I walked git log per path and read each fragment at the newest commit that touched it. For a consumed fragment that commit is the one that deleted it, so git show returns empty. My probe reported "0 commits touched" for six paths — a self-contradiction, since those paths came out of the log — and I only noticed because my total disagreed with the peer's. Their probe had the same defect and skipped 115 of 127 silently. Two independent probes, the same blind spot, because it is a property of the directory and not of either implementation.

What settled it was changing instrument rather than fixing the query: git rev-list --all --objects enumerates every blob ever reachable and git cat-file reads one by hash. There is no path-to-commit lookup, so there is nothing to get wrong — 129 paths, 239 distinct versions, unreadable: 0, and the count fires twice. A probe that cannot report what it failed to read is worth less than one that reads less and says so.

The last step was the one that nearly went wrong in the other direction. Two instances looked like enough to weaken the ruling, since the second fragment stamps neither heading. The obvious move was to set it aside as pre-convention — and that is false: 10 of 126 fragments have an unstamped first heading, scattered to the day before this was written. Stamping never became universal, so there is no cohort boundary to hide behind. It is ordinary non-compliance, it licenses nothing about second headings, and the ruling stands at n=1. The reason it stands is not the reason I first had.

The rule. Before generalising a count, name the population the number came from and ask whether the directory deletes. Where a probe can silently skip, prefer the instrument with no lookup to get wrong — and make it report what it could not read, because a zero and an unreachable set look identical in the output.

I read an error message as evidence of the arm that raised it (20260902 02:04)

I ruled that closing a hole in a fragment gate was not new process, because "the sibling stream already gates this". The evidence was three filenames through changelog.d: the canonical one passed, a typo'd prefix and a prefix-less one were both refused, and the refusal said "filename must be YYYYMMDD_HHMM-<category>-<slug>.md". Prefix gate, plainly.

There is no prefix gate. It is a category gate, and one more fixture separates them: fixed-a-thing.md — no prefix at all, but its head is a category — is accepted, while banana-a-thing.md is refused. A malformed prefix is refused there only incidentally, by shifting the first hyphen-separated token out of the six allowed categories. The message names the whole convention because the convention is what the author wanted the reader to go and read; it is not a statement about which arm fired.

Two refusals confirmed a mechanism that one more row would have falsified, and the missing row was cheap and obvious in hindsight: hold "prefix-less" fixed and vary only whether the head is a category. I varied two things at once in every fixture I wrote, so no comparison in the set could isolate either.

I have a rule that a null result proves nothing until the selector is shown able to fire, and I applied it that same hour to someone else's work. This is its mirror and I did not recognise it: a refusal proves nothing about which arm refused. A firing selector and a firing gate are the same problem — output that is consistent with the hypothesis is not evidence for it unless something in the set could have come out the other way.

The peer who found it did the thing I had not: constructed the case where the two hypotheses disagree, rather than adding another case where they agree.

And a second slip in the same edit, from the same family. I wrote 20260902 00:35 into three places, composed from my previous clock read rather than taken from the clock. The machine had slept about ninety minutes; the real time was 02:04. Every stamp I write carries a rule saying read the clock, never compose it — and the failure mode is not forgetting the rule, it is believing you already know the answer well enough not to check. Which is exactly what the error message did to me one paragraph up.

The rule. When output is consistent with what you expect, ask what the other hypothesis would have produced, and go and build that case. If nothing in your fixture set could have come out the other way, you have collected agreement, not evidence.

A method is not a measurement point, and --all is not a corpus (20260902 02:45)

I corrected a count by naming a better instrument and thought that finished the job. It did not. The first version of row 14 said 116 fragments, exactly one, measured over the live directory and stated as a fact about the corpus. The correction replaced it with 129 paths, 239 versions, taken from the object store — the right instrument, and the row said so: git rev-list --all --objects.

Re-checking my own branch before landing it, that number reproduced against nothing. --all is not a corpus. It is every ref any branch happens to hold at the instant it runs, so it counted unlanded work on my branch and on a peer's, and it had drifted to 132 paths / 243 versions inside an hour — without a single fragment being written to main.

The tell was visible in the row and I had not looked: three denominators for one directory. Row 14 said 129 and, four sentences later, 10 of 126. Row 17 said 131. One of those was all-refs, one was main, one was a moment. A row whose whole argument is that a count must state its population was carrying a numerator and a denominator drawn from two different ones.

Pinned to origin/main at 7751f96, landed history only, everything reconciles: 126 retro.d paths, 233 versions, changelog.d 177, combined 303. Both of row 14's denominators become the same 126. The conclusions all survive — still exactly two two-heading fragments, still the same two files, still 10 of 126 unstamped at 7.9%, still 0 of 52 unprefixed stems beginning with any digit. Only the totals were wrong, and only because they had no anchor.

The rule this leaves: a measurement is reproducible when it states an instrument and a point — a ref and a sha, not --all and not HEAD. Prefer the landed history as the population, because it is the only one that does not change when someone else commits. And when a single row carries two numbers about the same set, check they came from the same run before checking they are right: I verified the instrument and never verified that the two halves were counting the same things.

The same defect, three times in one day, each time caught by someone re-running rather than re-reading.

The one surface a later pass cannot fix is the one nobody reviews (20260902 07:34)

MEDIUM — three pushed commit messages on this branch carry four false sentences between them, and a commit message is the only artefact here that cannot be corrected in place. Every review pass on this branch has read the diff. Pass 9 read the messages, and found that the surface holding this increment's reasoning — the surface a future reader reaches for when asking why a line is the way it is — had never been checked by anything.

commit	the sentence	measured
0e424c7	markdown-links: 113 files, 375 links	the gate on that exact tree says 114 files. The link count is right
2914b09	"All five test names they name resolve"	the three docs/VERIFICATION.md cells that commit added name 11 distinct tests
2914b09	"startswith("## ") shipped in eight of them and the pass-3 lookaround in three"	over the population it names — every commit from 50dd420 to 6bc8078 — 5 and 2
f1de4c1	the mutant "is killed by the negative assertion rather than the positive one"	it dies at the positive one, tests/test_fragments.py:1005, and pytest never reaches the negative

And the pass that wrote this fragment put two more onto the same unfixable surface, within the hour. Both were caught by re-reading my own diff before landing, which is the only reason they are here as a record and not as a fifth and sixth row somebody finds next week:

commit	the sentence	measured
f48e1cf	the unclosed-<div> fixture renders "one <h2> where the source has four"	the source has three ## headings. The four is anchors_of's count, which includes the # heading — two instruments, one number, and I carried the number from the wrong one
bc68475	"four of them carry a false sentence", of commit messages	four sentences across three messages; 2914b09 carries two. The table underneath it has always been right, and the sentence summarising the table was not

Neither is a hard number to check. Both were written in a commit whose subject is other people's false counts, and both survived the writing of that commit. A pass that is looking outward does not look at itself, and the cheapest defence found so far is a mechanical one: before landing, re-read your own diff for every number in it and re-derive each, treating your own prose exactly as you would treat a claim raised by a lens.

The fourth is the one with a lesson beyond arithmetic. The false sentence was in the test's own docstring first, and the commit message copied it — "The negative assertion is the load-bearing one", written from reading the two assertions rather than from applying the mutant to them. Both would fail against a stripped message; only the first one reached reports. So the claim was not a slip made in a hurry at commit time: it was a plausible reading of the code that nobody executed, duplicated into a surface where it could no longer be edited. The docstring is corrected in this increment; the commit message is not correctable.

The third is the sharpest, because it was corrected once already and not by that route. 04497a9 replaced the selector — "every commit from 50dd420 to 6bc8078" was a range nobody had examined — and re-measured over full history, getting 4 and 1. It never re-derived 8 and 3 over the original population, so it left a false pair standing while reporting that it had fixed the sentence containing them. Replacing a claim's population is not checking its numbers. Three populations here, three answers — 8/3 written, 5/2 true for that population, 4/1 true for the one that replaced it — and only the first is wrong.

LOW — and one of the thirteen findings did not survive being checked, which is the reason the rule exists. A reviewer reported that 6bc8078 had replaced a correct census — pass 5's "28 of the tree's 30 battery sections carry the prefix" — with a false 27/19. Measured at 5329d24^, the tree pass 5 was describing: 30 sections, 27 carrying (19 unreleased, YYYYMMDD · and 8 X.Y.Z ·). 6bc8078 was right. Pass 5's 28-of-30 is the pair that matches no single selector.

And the reason it matches none is a section every census so far has silently dropped. Count a section as a comment line sandwiched between two # ───── rules and there are 30. Accept ASCII hyphens as rule characters too and there are 32 — because tools/batteries/src-pinakes-pairing.toml:369 draws its rules with - and writes # unreleased, 20260831 - S16's residue…, a hyphen where the convention has ·. It is non-conforming twice over, it predates this branch, and it is still non-conforming at this branch's tip. Under the loose selector the pre-fix tree reads 28 of 32. So pass 5's 28 is right about one population and its 30 about the other, and the two numbers in one sentence come from two different questions.

This is one class with the fragment landed beside it, not two. retro.d/ also carries A method is not a measurement point, and --all is not a corpus, written the same night about the same defect — a number stated without the selector that produced it. Read them together: that one names the rule, this one is the rule failing in the hands of the reviewer applying it.

This started as a bare path in prose, because when it was written the branch was twelve commits behind the file it cites and a link would have been red on its own gate. Green at the moment of landing beat the better reference. It is a link now that both are on main, and the reason is not tidiness: tools/markdown_link_gate.py reads retro.d/, so a path in prose is unchecked text and a link is a gated reference — this one goes red if either fragment is ever renamed, and the prose version would have gone quietly stale. Ruled by the planner.

The lesson is not "measure more". All three parties — the writer, the corrector, and the reviewer who called the correction wrong — ran a count. Each ran a different count and none said which. A census of a corpus with two spellings of its own delimiter has two answers, and the one you get is chosen by the regex you happened to write. Naming the selector is what makes two measurements comparable; it is also what makes a disagreement resolvable instead of a standoff.

What to do about a false sentence that is already pushed. It cannot be amended — the branch is shared and the sha is cited in this fragment and in docs/VERIFICATION.md rows. So the correction lives here, in the retrospective, which is the only append-only record of the same events. That is weaker than a fix and it is what is available. The stronger move is upstream: a commit message asserting a count is asserting something checkable, and nothing on this branch checked one until the ninth pass. Reading the message alongside the diff costs one command and would have caught all four before they were pushed.

The census I called correct was the one mixing selectors (20260902 07:41)

Reviewing a branch's commit messages, I raised that 6bc8078 had replaced a correct census with a false one: an earlier pass had written "28 of the tree's 30 battery sections carry the prefix", and the later commit said 27, of which 19 and 8. I filed it as a regression — a right number made wrong.

The coder re-measured and refuted it, and I re-derived it independently rather than accept the refutation on report. At 5329d24^, over 11 battery files: counting sections delimited by box-drawing # ──── rules gives 30 total, 27 carrying (19 + 8), 3 not. Also accepting ASCII # ---- rules gives 32 total, 28 carrying, 4 not.

So 6bc8078 is exactly right under the strict selector. The sentence I called correct — 28 of 30 — is the one that matches no selector at all: its 28 comes from the loose count and its 30 from the strict one. I had promoted an incoherent pair over a coherent one because the incoherent pair was older, and "a later commit changed a number" is a shape that reads as regression before it is checked.

Neither census stated its selector, and that is the whole defect — the same one I had landed a retro fragment about ninety minutes earlier: A method is not a measurement point, and --all is not a corpus. I wrote the rule and then failed to apply it to the very next thing I looked at, in a review whose purpose was to catch exactly this.

Why the selector is load-bearing here rather than pedantic: the coder found tools/batteries/src-pinakes-pairing.toml:369 uses ASCII rules, and across tools/batteries/ there are 56 box-drawing rules against 6 ASCII. A ─-delimited census is therefore structurally blind to that file. The two counts are not rounding of one another; they see different trees.

The rule this leaves: when two measurements of one set disagree, do not assume the older is the baseline. Ask which selector each used first — the disagreement is usually the population, not the arithmetic, and the number that survives is the one whose selector was stated.

I armed a CI watcher on a sha I had composed, not read (20260902 08:11)

Three consecutive tips had their CI verdicts lost to supersession, so I armed a bounded watcher on main's tip and told the user I would land nothing until it reported. It polled and reported nothing. I read that silence as correct and said so in as many words: "My bounded poll has produced no output yet, which is correct — it only writes when every workflow on that sha reports."

It was not correct. The watcher was polling 6a9245aca7bb5aeb0d0b30ba1c4ab4a5bd9b5cbb. The tip is 6a9245aff04805cff66ed3d1c4f17d72b6fab4dc. The two agree on the seven characters I had in front of me and diverge on the thirty-three I did not. I expanded a short sha into a full one by writing plausible hex. gh run list --commit <that> returns [], the loop's [ -n "$row" ] guard never passes, and the watcher runs to exhaustion printing an empty verdict block. It could not have reported a red build. It could not have reported anything.

The verdict arrived from the coder's watcher instead, and I verified it myself: run 33606706108, CI completed/success, 15 of 15 jobs green, headSha matching the real tip. So the outcome was fine and the instrument was worthless, which is the worst pairing — nothing about the result told me the tool was broken. I found it only when stopping the task printed the command back to me.

Two failures, and the second is the bad one.

The first is composing an identifier instead of reading it — git rev-parse was one call away, and the repo already carries this rule for timestamps (read the clock, never compose it). It generalises: a sha, a run id, a version, a line number. If it identifies something, read it from the thing.

The second is that I had a null result and treated it as a measurement. Zero rows from gh and zero output from the watcher are the same shape as not finished yet, and I picked the reading that matched what I expected. A selector that has never been shown to fire cannot report absence — this is written down, and I had cited it in this very session while reviewing someone else's work.

What would have caught it, cheaply: before arming any watcher, run its query once in the foreground and confirm it returns a non-empty row. A watcher whose first poll matches nothing is not waiting; it is misaddressed. That check costs one command and is the difference between a hold that means something and a hold that is decoration.

This is the third instance in one night of the same family: a status list read without asking what it was capable of containing (a false RED), a workflow absent rather than cancelled because its path filter never matched, and now a query addressed at nothing. In all three the empty or partial answer was read as information about the instrument's world rather than about the instrument.

A postscript, because it happened while this file was being written. Investigating the above, I found two landed fragments linking a sibling by filename — forbidden by retro.d/README.md because the path stops resolving once everything is spliced into one document. I proved it (make docs exit 2, two strict-mode warnings naming that link), fixed both, and then ran git checkout -- retro.d/ to undo the splice. That restored the two fragments to HEAD, discarding the fix I had just made, and deleted this file, which was untracked and had been consumed by --apply. The repo warns about exactly this — commit before mutating; git checkout restores to the last commit, not to the pre-mutation state. I had also been told the count: ls retro.d/*.md | wc -l printed 24 where 25 was correct, and I read the number without checking it against what should have been there. A count you do not predict before you read it cannot surprise you, which is the same defect as the watcher — an instrument consulted with no expectation attached reports nothing either way.

And a fourth instance, inside the fix for the third. Rewriting those two links needed the sibling's anchor, so I computed it "with the renderer that builds the site" — and wrote exactly that sentence into b4fa8e8's commit message, where it stands, false. What I actually built was a markdown.Markdown from mkdocs.yml's markdown_extensions list. That is not the site's renderer: mkdocs.yml installs GitHub's slug algorithm through mkdocs_hooks.py, and says so in a comment eight lines above the toc block — "neither Python-Markdown's default nor pymdownx's matches it". My two greps windowed on markdown_extensions and on - toc, and between them excluded the one comment written for somebody doing what I was doing. Python-Markdown's default collapses runs of hyphens; GitHub's does not, because it discards the backticks and keeps the spaces around them. The anchor I wrote was …-and-all-… where the site emits …-and---all-…. The coder caught it, having made the identical mistake an hour earlier and reported the identical wrong string to me as verified.

Two things kept that one invisible, and both are worth naming. A code span is not a link, so make docs returns 0 whatever the string says — the fix for a link that fails loudly was checked by a build that cannot see the replacement, which is a seam I introduced in the act of closing one. And neither of us lacked an instrument: we each had one, ran it correctly, and it was one config short of the thing it claimed to reproduce. A reproduction is a claim about coverage, not about effort. Corrected against tools/markdown_link_gate.py's github_slugify() and against the built site/RETROSPECTIVES/index.html, which agree — with a control that the non-collapsing is systematic rather than a quirk of this heading: three ids on that page contain ---.

A fifth instance, three hours later, and this one is the mirror of the first. Checking whether two of the morning's tips had passed CI, I ran gh run list --commit 5654cc4 and gh run list --commit 293d434. Both returned []. For about a minute I believed two landings had never triggered CI at all. gh run list --commit requires the full 40-character sha; an abbreviated one — valid to every git command in the repository, and the form the tool itself prints in gh run list output — matches nothing, exits 0, and says nothing. The control settles it: gh run list --commit $(git rev-parse 5654cc4) returns two runs. So the first failure in this file was an identifier I composed and this is an identifier I read correctly, in a form the tool rejects silently. The rule that survives both is not "read the sha" but "prove the query fires." git rev-parse before any --commit, and the first poll is still the assertion.

A sixth, and it is a defect in the alerting rule I wrote, not in a command. My standing CI-watcher guidance alerts on failure, timed_out, startup_failure and action_required, and deliberately excludes cancelled as "normally supersession." Under that rule this morning was silent, and this is what it was silent about:

5654cc4  07:55:49 -> 07:58:23  push/cancelled     the retro heading + stamp gate
4e13d6f  07:57:47 -> 08:03:34  push/cancelled
6a9245a  08:03:31 -> 08:08:27  push/success

.github/workflows/ci.yml:8-10 sets concurrency: group: ci-${{ github.ref }} with cancel-in-progress: true, so every push to main kills the run before it. Two consecutive tips — one of them a new gate's own landing — got no verdict, 2m34s and 5m47s in. Coverage did hold: 6a9245a is a descendant of both, ran 15 of 15 green, and 293d434 later did the same. But it held by ancestry, and ancestry is not what the watcher was checking. Nobody asked. I had written "three consecutive green tips" into my own resume file naming the three that completed, without noticing that between them sat two that had not.

cancelled is not a state, it is two states wearing one word: a superseded run whose commit a later completed run contains, and a lost verdict whose commit nothing has yet certified. Telling them apart costs one ancestry check — is there a completed, successful run whose sha has this commit as an ancestor? — and without it the exclusion converts an unverified tip into silence. That is the same defect as every other one in this file: an instrument that cannot distinguish "fine" from "never looked", consulted with no expectation attached. It is worse here than in the others, because this instrument's whole purpose is to be trusted while nobody is watching, and because I am the one who wrote the exclusion into the rule.

A seventh, and it is the sharpest one here, because I built it while fixing the sixth. Twenty minutes after writing the paragraph above, I armed a v2 watcher whose whole purpose was to stop reporting cancelled as if it were nothing. It ran, polled thirteen times, and printed this where the verdict belonged:

SyntaxError: unexpected character after line continuation character

python3 -c 'import json,sys; [print(f" {r[\"workflowName\"]} …")]' — inside a single-quoted -c, the shell passes the backslashes through, and {r[\"workflowName\"]} is not valid inside an f-string. Every other query in the same script used --jq and worked; this one line reached for Python because I wanted two fields on one row. The rest of the script survived: the job count printed 15, the non-success selector printed nothing, and the run really was green — confirmed independently, gh run list giving completed/success and the job list's non-success set empty.

So the shape is exact: the instrument written to stop an instrument reporting falsely, reported falsely. And it failed in the one direction that is hardest to notice — it suppressed the line naming the workflows and their conclusions, which is the line the cancelled finding was about, while still printing a reassuring 15 underneath. Had a run actually been cancelled, the discharge check would have run and its output would have sat directly below a stack trace I might have skimmed past on the way to the green number.

The rule this file has been circling, stated plainly at the seventh attempt: a checking tool needs its own control, and the control is not "does it run" — all seven of these ran. It is make it print the thing you already know. One green sha through the watcher before arming it; one known-red mutant before believing a mutation run; one heading you have already resolved before trusting a slugifier. Each of the seven would have died in under a minute against a case whose answer I held. I have written that sentence three times tonight in three different registers and then not done it, which suggests the failure is not knowing the rule.

The config does not name the renderer (20260902 08:30)

A peer replaced a forbidden link with an anchor and asked me to check the string. I built a renderer from mkdocs.yml, ran the heading through it, and reported the anchor correct. It was wrong by two hyphens, and I had certified it.

The site's slugifier is not in mkdocs.yml. mkdocs_hooks.py's on_config installs _github_slugify onto the toc extension at load time, so markdown_extensions — the list I read — describes every part of the renderer except the one part the question was about. Measured against the heading in question:

instrument	result
mkdocs_hooks._github_slugify — what the site runs	…-and---all-is-not-a-corpus-…
Python-Markdown's default toc slugify — what I ran	…-and-all-is-not-a-corpus-…
pymdownx.slugs.slugify	A-method-… — keeps the run, keeps the capital

Two of the three reconstructions are wrong, in two different ways. Only the hook is right, and the hook is exactly what is not declared. The repository already knew this: mkdocs.yml lines 86–88 at 6a9245a say "The slugifier is installed by mkdocs_hooks.py, not named here", three lines above - toc:.

The warning was inside the region I read, and my reader could not represent it. I loaded the block with yaml.safe_load, and a YAML parser discards comments by construction — the one sentence written for this mistake was invisible to the instrument making it, not somewhere else in the file. A peer reconstructing the same string independently windowed two greps from markdown_extensions to - toc and excluded the comment between them. Different reader, same blind spot, same wrong string. For about an hour we treated that agreement as corroboration. Two instruments built from the same incomplete source do not check each other; they repeat each other.

One correction I owe my own notes: I first recorded the cause as a SafeLoader subclass I had written to stub !!python/name: tags past a ConstructorError. Checking it to write this, it is not — those tags carry the emoji index, and the slugifier was never in the YAML for a stub to drop. The stub was a near miss, not the mechanism: it is the moment I started patching around the config's irregularities and read them as noise.

The rule is cheap and I had it available the whole time: when you need the site's renderer, build the site. make docs, then read the id= attribute out of the built HTML. That is what settled it in the end — the correct anchor read off site/RETROSPECTIVES/index.html, with a control confirming the two-hyphen string appears nowhere among the 904 ids on the 30 pages built at 8d71489. A null result carries no information until the selector is shown able to fire, and a positive one carries none until the instrument is shown to be the one that ships.

This is the same defect as A method is not a measurement point, and --all is not a corpus, one layer down: not a number measured over the wrong population, but a string measured with the wrong program. The sentence that names it is the anchor to that fragment, which is the string I got wrong.

Three more in the same hour, all caught before they cost anything, all the same shape. A --check run through grep -v returned an empty FRAGCHECK_EXIT=. A nohup ./check.sh > log 2>&1 & discarded the gate's status entirely, leaving me about to read "all gates green" off stdout and call it a gate — a backgrounding form eating the exit status is the pipe rule wearing different clothes, and it is not a trap peculiar to nohup. And a public-repo secrets vet used a lookahead ugrep rejects: the pattern errored, the || branch fired, and it printed "clean: no paths, addresses, secrets or external URLs" — a false all-clear from a check that never ran, on the one gate where a false clean is worst. Each was found by asking the same question the anchor needed: not what did it say, but was it able to say anything else.

The restore command is the one that eats the work (20260902 08:53)

I proved a change by running the thing it changes: splice both fragment streams, build the site, read the anchors out of the built HTML. Then I ran git checkout -- . to put the consumed fragments back, and destroyed my own uncommitted increment — the gate, its nine tests, and four link restorations. git status came back empty and that was the whole report.

The rule is already written, in the mutation section: "git checkout <file> restores to the last commit, not to the pre-mutation state, and silently reverts uncommitted fixes." I had read it. What it did not stop is the reason it did not stop me: the operation felt like a read. Splicing and building are a measurement, and the command that undoes a measurement does not present itself as a write. A peer did the identical thing four hours earlier on the same night, lost a fragment, and rewrote it. Two agents, one command, both having read the rule — that is a defect in the remedy rather than in either of us. The remedy that would have worked is one word longer: commit before the measurement, not before the mutation, because a splice is a mutation whatever it feels like.

Recovery cost about ten minutes, and only because I had copied retro.d/ and the gate to scratch first — for reasons unrelated to this, and not the test file, which I rewrote. Everything was then re-verified from the restored tree rather than from the earlier run's output, since the earlier run had measured a tree that no longer existed.

A second instrument failure the same hour, and it is the sharper one. A peer and I each counted the live ](#…) links in docs/RETROSPECTIVES.md. We both got 2. The population is 3. Their pattern, \]\(#[a-z0-9][a-z0-9-]*\), misses #measure_sync_cpupy-… because of the underscore. The pattern I attributed to them — a grep quoted inside another fragment — misses #design-review-passes-17-pre-implementation because it requires a -YYYYMMDD-HHMM suffix. Two selectors, two unrelated blind spots, one agreeing wrong answer. Agreement between instruments is worth nothing until the instruments are shown to be independent, and these looked independent precisely because neither of us had written the other's regex down. Earlier the same night the same pair of us computed the same wrong anchor by the same wrong method and read that as corroboration too; this is the harder version, because here the methods really were different.

And a third, which is the one that generalises. A peer measured a line at 254 characters and dismissed two neighbouring lines as "2 characters over, not worth a commit". The line was 252 characters and 254 bytes — awk length counts bytes in this locale — and the two neighbours were 99 and 100, never over at all. The headline survived the error: at 252 or 254 that line was two and a half times the file's width and had to be fixed either way. That is exactly why the error was safe to keep. A measurement can be wrong in a direction that does not touch the conclusion and still poison every judgement made beside it, and those judgements are the ones nobody re-checks, because the headline held. The number the conclusion rests on gets scrutinised; the numbers riding along beside it do not.

The find worth keeping came from reviewing the change after it was green. A stream README.md is excluded from the fragment set — its own links resolve where it sits — and it must also be excluded from the anchor universe, because its headings never reach the spliced document. Two exclusions, two functions, and only the first had a test. Without the second a fragment could link #how-to-write-a-fragment, pass on the branch, and die at the release cut: the exact failure the new gate exists to remove, produced by the gate. Nothing about it was visible in the diff, and the suite was green when I went looking.

The fix for a loud crash was a silent deletion, and only a probe said so (20260902 09:27)

S1 read as a one-line fix: hash_file lets PermissionError escape the walk, so catch it and skip the file. The crash is loud, the traceback is ugly, and skipping is obviously kinder.

It is not. pair() decides what to do with an indexed document by whether the walk still reports its path — absence means gone. A walk that skips an unreadable file therefore emits SoftDelete: the row retired, its chunks deleted, pnk sync printing 1 removed and exiting about something else entirely. A chmod 000 would have removed the document from search. The obvious fix turns a loud crash into silent index loss, which is the shape this repository ranks worst, and it would have shipped green.

Nothing in the plan said this. It was found by writing eleven lines of throwaway script that built an IndexSnapshot and a WalkSnapshot by hand and printed what pair() returned — before a line of the fix existed. The whole finding cost one tool call.

The rule that generalises: when a fix changes what a downstream consumer is told, ask the consumer before writing the fix. Not after, and not by reading it. Reading pair() would have worked too — the vanished-path loop is right there — but reading is what produced the one-line plan in the first place. The probe answered a question I already believed I knew the answer to, which is the only kind of question worth spending a tool call on.

The fix's own first output was a command that destroys a ULID

Running it printed orphaned sidecar (kept; remove with pnk doctor --prune): docs/c.md.pnk.yaml for a document sitting on disk, readable sidecar and all. _orphans derives orphanhood from the walked file set, and the unreadable document had just left it. Following that remedy deletes the sidecar, and with it a permanent id that other KBs may link to.

It was found by running the fix on a real KB with the real [light] backend, not by reading the diff — the diff is three lines and every one of them is correct. This is the seam rule paying out: unit tests prove a component honours a parameter, and only running proves what the user is told.

An exit 0 from an instrument that never reached its population

pnk doctor was checked against an unreadable document and came back clean — exit 0, no traceback. The conclusion "doctor is fine" was one keystroke away and would have been wrong.

The row had been marked extraction_backend='claude', and the only paid backend name is claude-vision. So recorded_is_paid was false, the loop body never ran, and the unguarded hash_file was never reached. The tell was in the output and easy to skim past: paid extraction not requested: none. With the real name substituted, doctor died in a PermissionError traceback.

That is the fourth time in two days a null result nearly became a finding here. The others were a grep pattern with a lookahead the tool rejected, a sed range using \b that BSD sed does not support, and a grep 'walk_sources(' over tests/ that returned nothing because the call site aliases it to real_walk. Same shape every time: the instrument could not reach what it claimed to have checked, and its silence read as an answer. A control that must fire is the cheapest insurance in this repository, and it is still not automatic for me.

A gate against unnoticed drift, and the two shapes it nearly could not see (20260902 10:07)

Row 15 was recorded as one comment on one line: a battery section header reading # 20260823 · … where the reserved forms are a version or unreleased, YYYYMMDD. The header itself had already been fixed. What was left was the gate — refused once, while the question of whether adding a check counted as new process was open, and unblocked when that question was answered.

Writing it produced two findings, and neither was the one the row described.

The audit could not see the second instance, because of the shape of its own selector. Row 15 concluded this was "the only one matching neither". It was not. src-pinakes-pairing.toml carried # unreleased, 20260831 - S16's residue: … — a hyphen standing exactly where the · separator belongs. The audit had searched for headers containing ·, so the one other non-conforming header in the tree was the single shape its query could not return. The gate found it on its first run. A selector defined by the correct form cannot find a violation that consists of that form being absent. The row's other numbers fail the same way, and its own arithmetic said so without any re-measurement: it reported 30 sections, 19 unreleased, YYYYMMDD ·, 8 X.Y.Z ·, one matching neither — and 19 + 8 + 1 is 28, two short of its own stated total. Measured at a4077b5, the commit that wrote it, the tree held 30 sections: 20 unreleased, 8 version, two matching neither. One invisible header is missing from exactly two tallies, which is exactly the two the row was short.

And I made the neighbouring error while correcting it. My first correction tabulated 36 sections, 26, 9, 2 against the row's 30, 19, 8, 1 and called the total wrong. It was not: the tree grew from 30 to 36 between a4077b5 and today, and the row's 30 and its 8 were both right for the tree it was measuring. What was wrong was 19 and only one. Comparing today's tree against a claim about a past tree turns growth into error and hides the real defect — so a count correction has to name the commit it was measured at, not just the selector.

Then the gate itself nearly shipped with the identical defect. Battery sections are fenced by horizontal rules, and the first RULE regex matched only the box-drawing U+2500. Nine batteries use it. Two use ASCII hyphens — and one of those two was src-pinakes-pairing.toml. So the gate read 34 of the tree's 36 headers, reported a clean sweep, and the file it skipped was one of the two that had drifted. A gate written to catch unnoticed drift, silently skipping the drift, on its first run, in the increment whose whole subject is a selector that could not match its target.

It was caught by refusing to accept a clean result. non-conforming: [] is a null result, and a null result carries no information until the instrument has been shown able to fire; counting the headers found against the headers on disk is what turned [] from a pass into a discrepancy. The same discipline had already been spent twice this increment — on a search assertion that passed for a word in no document, and on a git cat-file lookup where zsh ate :t out of "$tag:tools/x.py" as a history modifier and reported a file absent from a tag that contains it.

What generalises: a checker's own matcher is part of what it checks, and it is the part nobody tests. The assertions were right in all three cases. What was wrong was the set they ran over. So the gate now counts both fence shapes, says in a comment why rather than leaving a magic alternation, and the docstring names what it deliberately does not check — calendar validity, and section ordering — because a gate that quietly grows a second subject is how the next selector ends up not matching the thing it is looking for.

I published a count I never ran, and a peer nearly stopped a correct release (20260902 10:26)

I reported the 0.32.0 splice as "12 changelog + 26 retro". The tree held 13 and 27. The release was correct; the number in my status message was not, and I had not measured it — I read it off my own earlier prose rather than off git ls-tree.

What it cost. The coder reasoned correctly from my figure and reached a serious conclusion: if one fewer of each was spliced, the missing pair is exactly S1's, and 0.32.0 would ship the fix while describing nothing. It messaged "stop before the tag". The tag was already pushed by the time the message arrived, so the cost was a verification round rather than a release — but the reasoning was sound and only the input was wrong.

Two commands settled it, both of which I should have run before quoting a number:

git ls-tree --name-only v0.32.0 changelog.d/ retro.d/ | grep 20260902_0927   # empty
git show v0.32.0:CHANGELOG.md | grep -c unreadable                          # 18

The lesson is not "count more carefully". It is that a number inside a status report is a claim with the same standing as a number inside a document, and it travels further and faster because peers act on it directly. This repository already rules that a claim resting on a set you selected must state its selector. A count in a message has a selector too — git ls-tree <ref> <dir> — and naming it is what makes the claim checkable by the person receiving it.

What the coder did right, and it is the reusable half: it said plainly that it could not verify the branch (it was not on the remote), declined to assert the defect, and handed me the two commands that would settle it either way. A well-formed doubt with its own falsification attached costs sixty seconds. The same doubt asserted as a finding would have cost an argument.

The gate I ran was not the gate CI runs, and the gap is where the defect was (20260902 10:26)

./check.sh exited 0 on the release branch. main went red on the same tree.

Not a flake, and not a different commit. My local run skipped 128 tests because this machine has neither the [pdf] nor the [claude] extra installed, and CI's check job is a three-leg matrix over [light], [light,pdf] and [light,pdf,claude]. The failure was in tests/test_pdf_trace.py — inside the skipped set, by construction.

The defect it found was real and had been dormant for a month. A test asserts reservation.cost_eur == estimate.per_request_eur, and that comparison straddles the ledger's write-time quantisation: accountant.py:193 multiplies the EUR estimate back by the rate, ledger.py:139 quantises on write at 1e-6 ROUND_HALF_UP, ledger.py:127 divides back on read.

rate	per_request_eur × r	quantised	read back	equal
1.1596 (seeded)	0.3535000000000000000000000000	no-op	…3083822	yes
1.08 (the original seed)	0.3535000000000000000000000000	no-op	…8148148	yes
1.159 (2026-09-01 ECB)	0.3534999999999999999999999999 — one short	snaps up	…5452977	no

So the assertion had never held by construction. It held because two successive rates happened to make the round trip land exactly on the quantum. Refreshing that constant — a step docs/RELEASING.md requires at every release — is what falsified it.

And I published a different mechanism before checking this one. I read cost_eur = cost_usd / usd_per_eur, assumed cost_usd was the summed USD, and wrote up an arithmetic-ordering story — (a/r)+(b/r) against (a+b)/r — in three documents and a commit message. It is a coherent account that predicts the observed digits, which is exactly why it survived my own reading. A peer reproduced the real route and refuted it. The fix I recommended on the strength of it would have removed the …9999 from this fixture and left the assertion false for 66% of a 40 000-case sweep; restricted to requests == 1 it does hold, which is the trap, because the fixture is requests == 1 and so is any sweep someone writes to check it.

The rule I am taking from that: an explanation that fits the numbers is a hypothesis, and the discriminating step is reading the code path that produces them, not the code path that would.

Three things follow, and only the first is about me.

A skip is not a pass, and a green local gate is a claim about the legs that ran. ./check.sh prints its skip count; I read the exit status and not the count. The honest form of "green locally" is "green on the one leg my machine can run".

A test whose truth depends on an unmentioned constant is not pinned to what it claims. Nothing in test_pdf_trace.py names usd_per_eur, so nothing warned that changing it could break the assertion — and nothing warns now that a future refresh could make it pass again for the same bad reason. A second rate in the test is what turns a coincidence into a pin.

The order of discovery is worth keeping. The artifact was verified — installed from the index, wheel opened, subject found inside it — before CI went red. A verified artifact and a green main are two claims, and it is possible, as here, to have only the first. Neither substitutes for the other, and a release note that reports one as though it were both is the error this record exists to prevent.

Both hops were green for the value of a constant neither of them mentions (20260902 10:45)

Two assertions in tests/test_pdf_trace.py, one live and one latent, were correct against every input anyone had ever run them on. Neither was a badly written test. Both depended on the value of a constant that appears nowhere in their text, and this repository refreshes exactly that constant on a schedule.

hop	what it compared	why it was green	what ended it
2, reservation	cost_eur against estimate.per_request_eur	at usd_per_eur = 1.1596 the multiply back landed exactly on six decimals, so quantisation was a no-op	the 20260901 refresh to 1.159
4, reconciliation	a bare Decimal.quantize() against the ledger's ROUND_HALF_UP	today's prices make per-token USD exactly six decimals, so there is never a tie to resolve	nothing yet — it is latent

The trigger is scheduled, not hypothetical. docs/RELEASING.md § Before you start step 3 refreshes prices.toml at every release. That step is the input to both assertions and is named in neither. A price-granularity check — something that asks whether a refreshed rate or per-token price still leaves the arithmetic on the quantum — is arguable on this evidence. It is deliberately not built here: that is new process, and new process is not an implementer's to invent.

Measured, because the plausible story was wrong twice. A rearrangement that divides once instead of twice looks like a principled fix; over 40 000 randomised (rate, cost, requests) cases it fails 54% of the time — and 0.00% at requests == 1, which is the fixture's own shape. It would have passed this test, and any sweep shaped like the fixture, while leaving the assertion structurally false. My own first proposal failed 1.81% of the same sweep, because it inherited the very rounding-mode defect I then found in hop 4.

What the fix stops covering, said plainly. Both sides of hop 2 now go through the production quantise(), so it cannot fail on arithmetic: it pins the plumbing — that the reservation carries this estimate and not another number — and nothing about the arithmetic that produced it. Four mutants confirm the shape rather than assert it: an accountant that adds one quantum and a claude.py that reserves a different estimate field both die; PROMPT_TOKENS 700 → 701 and the ledger's rounding mode both survive. The survivors are not a gap to be closed but a seam to be named: the test recomputes the estimate from estimate_document, so any mutation inside that function moves both sides of the assertion equally and is structurally invisible to it.

And the word "pinned" does not apply to hop 4. Hop 4 is green today with or without the change, so no ordinary run can demonstrate it; it is a latent divergence closed before it fired, and showing it requires changing a price. Saying "pinned" would be claiming a failing test that does not exist.

A mutant killed for the wrong reason is a survivor wearing a green light (20260902 11:59)

The S4 battery came back 6 mutants, 6 killed, 0 survived on its first run. One of the six was not measuring what its row claimed, and the count could not say so.

The row is named "a backslash is left raw, so a Windows-style path parses as a name nobody typed". It removes the \\ entry from _TOML_ESCAPES, and its whole point is the quiet failure — a manifest that parses, that pnk doctor calls healthy, holding a name the user never typed. It killed through the eleven-value corpus, whose backslash value is C:\notes\kb.

That value cannot demonstrate the property. It contains \k, which TOML defines as nothing at all, so tomllib rejects the file:

tomllib.TOMLDecodeError: Unescaped '\' in a string (at line 2, column 23)

A parse failure — loud, and caught by every test in the file. The round-trip half the row is named for was asserted by nothing. Measured across three shapes:

value	without the backslash escape
C:\notes\kb	rejected — \k is not an escape
C:\notes	parses, reads back C: + newline + otes
C:\build	parses, reads back C: + backspace + uild

The fix was one test over C:\notes and repointing the row's kills at it. The mutant now dies on AssertionError: assert 'C:\notes' == 'C:\\notes' — an equality, which is what the row claims.

Three things worth keeping, none of them about escaping.

A clean aggregate is a fact about the harness before it is a fact about the subject. 6 of 6 killed has the same shape as a 30-agent verification panel that overturns nothing, and as a mutation run with no kills: consistent with the instrument working, equally consistent with it not discriminating, and the run cannot tell you which. Both happened in this repository on the same day, hours apart. The count is the last thing to read, not the first.

Read the kill reason, never the kill count. This was found only by reading the failure text of a mutant that was already dying. Nobody does that when the number comes back clean, which is precisely why nothing else could have found it: tools/mutate.py reports KILLED correctly, the anchor resolved exactly once, tests/test_batteries.py was green, and the row's prose was self-consistent. Every gate this repository owns was satisfied.

When prose names a value to demonstrate a property, run the value. The wrong example was written into a battery header whose own subject is parsing is the weaker claim — and it then survived being relayed to a second agent and dictated back as finished prose for a third file, still unmeasured. It was checked when it was executed, not when it was read, reviewed, or agreed. The general form is now scoped onto the queued row for the post-render validator: a remedy, a docstring, a --help string or a dictated paragraph that names a value to make a point is run, not reasoned about.

An example I reasoned about instead of running (20260902 12:00)

I dictated a sentence into tools/batteries/README.md claiming that without the backslash escape, C:\notes\kb renders valid TOML that reads back as a name nobody typed. The coder's mutation run falsified it before it landed. tomllib rejects that value — \k is not a TOML escape at all — so the mutant it described dies on a parse error, which is the opposite of the point the sentence was making.

The phenomenon is real; the example did not have the property. C:\notes parses and reads back as C: + newline + otes; C:\build parses and reads back as C: + backspace + uild. Three seconds of tomllib.loads separates them, and I ran it only after being contradicted.

Where it was going to land is the part worth keeping. That README's own subject is that parsing is the weaker claim, and the paragraph exists to teach the difference between a manifest that parses and one that round-trips. A worked example that fails on the parse side would have taught the opposite of the file's thesis, in the file's own voice, under a heading that made it authoritative.

The rule this repository already had did not cover it. A claim resting on a set you selected must state the selector is about populations. The command ran, the number was typed is about a figure copied out of prose instead of out of output. Neither reaches a value chosen to demonstrate a property, which is not a measurement at all — it is an argument, and it feels like reasoning rather than reporting, so nothing prompts you to execute it. The generalisation:

When a message, a header, a docstring, a remedy or a --help line names a value in order to demonstrate a property, the value is run, not reasoned about. An example is a claim with an executable form, and the executable form takes seconds.

The catch came from a mutation run, and only because someone read the failure text. The battery reported six mutants killed and none survived. One of the six was dying on a TOMLDecodeError while its row claimed a round-trip property that nothing in the suite asserted — so the row was green on a weaker property than its own name disclaims. A clean kill count concealed it; the kill reason did not. A mutant killed for the wrong reason is a survivor wearing a green light, and the aggregate that hid it has the same shape as a refuter panel that overturns nothing and a Blocked on cell nobody re-reads: a number that describes the instrument rather than the subject.

What the exchange did right. The coder held the dictated text rather than pasting it, said which claim it was refusing and why, and brought the run output and a three-row table of measured values. Refusing planner-dictated prose is the harder direction of the ownership split — the whole mechanism is content mine, keystrokes yours — and it is exactly the direction that has to work, because a dictated sentence arrives with more authority than it has earned.

S4's fix reproduced S4, on the one value class the framing excluded (20260902 21:18)

S4 was: a KB name containing a quote, a backslash or a control character wrote a pinakes.toml no parser could read, while pnk init exited 0 and printed created. pnk init refuses a directory that is already a KB, so the remedy surface was empty and recovery meant hand-editing TOML. The fix escapes every interpolated value for the TOML basic string it lands in.

Measured on the branch that carries that fix, through the real CLI:

pnk init /tmp/kb --name $'kb-\xff-name'
  → uncaught UnicodeEncodeError from Path.write_text (init.py:370)
  → /tmp/kb/pinakes.toml exists, 0 bytes
  → retry: "error: /tmp/kb is already a KB."

An unreadable manifest, init refusing the directory, an empty remedy surface. The same three sentences, on the branch whose whole purpose was to delete them.

The framing excluded the class, so no amount of care inside it would have found it

\xff on a command line is not valid UTF-8, and POSIX decodes it with surrogateescape (PEP 383) into U+DCFF — an unpaired surrogate. TOML's basic string admits %x80-D7FF and %xE000-10FFFF raw and skips the gap, and \uXXXX must name a Unicode scalar value. So the character has no TOML form raw and none escaped. It is not a hard value to escape; it is a value for which escaping is not the answer.

Every artefact of the increment said escape. The fix is an escape function. The tests are named for round-tripping. The docstring's "region this cannot reach" paragraph names TOML positions — literal strings, bare keys, numbers — and no values at all. The battery's six rows each removed one escape and watched a test die. Six mutants, six kills, and the whole apparatus was blind in the same direction, because the question it was built to ask was "is this escaped correctly", and the answer for this class is "there is nothing to escape it to".

The check that would have caught it is one question asked before the fix, not after: what is the set of values this promise is over, and does every member of it have an image? Enumerating the domain is a different act from testing the function. Eleven values were chosen for the corpus, all of them representable, by people looking for hard-to-escape characters.

The refuter was right about the mechanism and wrong about the disk

The reviewer that found this rated it medium. Its stated reason: "the crash happens before that file is written, so this does not brick the target directory — a retry with a valid name succeeds." Both halves are false. Path.write_text opens and truncates, then encodes, so the zero-byte file is already there when the encoder raises; and the retry is refused.

The refuter reproduced the traceback and reasoned from it to a state it never looked at — ls and wc -c were one command away. This repository's recurring failure has a shape, and this is it: a valid inference over a population nobody enumerated. The judge ran the command and overturned the severity upward, which is the only reason it was not landed as a medium.

Making a value legal relocates the question; it does not answer it

A second finding from the same review, measured and confirmed: pnk budget prints kb.name raw, so a name carrying an ANSI escape reaches the terminal as a live escape sequence, and a name carrying a newline breaks render()'s one-entry-one-line contract with no error anywhere.

Before the fix, such a name bricked the KB, so nothing downstream ever saw it. Escaping for TOML discharges the obligation TOML has, and discharges nothing that a terminal, a filename, a log line or an HTML page has. The moment a class of input stops being rejected, every consumer of that value inherits a question it was never asked — and none of them changed, so none was reviewed. The increment's own tests cannot see this by construction: they assert the value comes back out unchanged, which is exactly the property that delivers the bytes downstream.

The check is cheap and was not run: who reads this value now, and what does each of them assume about it? Here that is one grep and twelve call sites — eleven are JSON payloads or in-memory comparison, and budget/summary.py:193 is the single plain-text consumer.

One correction worth keeping, because the narrower claim is the defensible one: the review called this previously unreachable. It was not — a hand-written manifest carrying \u001b always parsed and always printed raw, on main too. The fix changed the route, from hand-edit your TOML to pass a flag. A widening, not a creation.

Code no test can pin, found by a count that came out wrong

The judge proposed excluding bool from the new allow-list: isinstance(True, int) is True, and str(True) is True where TOML's literal is true. It was built, and a test was written for it, and both were removed — because the escaper escapes content and never adds quotes, so a bool renders True from either branch: bare at dim = {{ embedding_dim }}, and inside the template's own quotes at name = "{{ name }}". The exclusion had no observable effect at any interpolation in the file.

What exposed it was not review. The fragment claims a red/green split, so the split gets measured by removing the hook and running the file. It came out 5 green where the four controls were known and named. One line of arithmetic, and the fifth green test was the new one — passing without the fix it was written to pin, with a battery row that would have survived. The arithmetic was the instrument; nobody read the test and saw it.

The review harness said "found nothing" when nobody had looked

The third adversarial pass over this increment returned, in its own words, "PASS FOUND NOTHING that survived refutation". All four of its lenses had stalled and errored; agents_done was 0. The verdict string was mine — the script tested survived.length === 0 and never asked whether anyone had looked, so a vacuous run and a clean run left by the same exit.

Pass 1 had already shown the softer version: 8 raised, 4 confirmed while five of its fourteen agents were dying on a session limit, one whole lens never running, and two findings never refuted — one of which was the only real code defect in that pass. Truncation is not random with respect to value: a lens that reads text finishes first, and a lens that must build a fixture and run a command reports last, so the expensive lens is first to be cut and it is the one that finds behaviour.

The fix is structural, not a habit: the harness records which lenses actually returned and reports VACUOUS / PARTIAL / CLEAN rather than a count. A count cannot carry the difference, and asking a human to read the error list first is asking them to distrust the summary they were given.

Which is the same lesson as this increment's earlier one arriving from the opposite direction: there, the aggregate was clean and the kill reason was wrong; here, the reason would have looked fine and the count was wrong. Keep a number in the fragment that has to be re-derived. It is the cheapest adversary in the repository.

A commit taken inside a mutation window shipped the mutant (20260903 08:30)

7193983 on the S4 branch has the subject "Retro: the review harness reported a clean pass while nobody had looked". It changed a retro fragment — and src/pinakes/template.py, by one line:

-        elif character != "\t" and (character < " " or character == "\x7f"):
+        elif False:

That is battery row a control character is left raw, live on disk, committed and pushed. Every control character in a KB name would have gone raw into a TOML basic string, which is the S4 defect this branch exists to delete, on the branch that deletes it.

The window is seven seconds wide and it is nobody's fault in particular

7193983 committed        20260903 02:46:58
template.py mtime        20260903 02:47:05     <- tools/mutate.py restoring, 7s later

The commit ran inside a mutation run. tools/mutate.py writes a mutant, runs the named test, restores the file; git add called anywhere in that interval stages the mutant, and the restore afterwards leaves the tree clean, so there is nothing left over to notice.

CLAUDE.md already says commit before mutating — because git checkout <file> restores to the last commit and silently reverts uncommitted fixes. That rule was followed here: the target was committed before the battery ran. The failure is its unwritten half, in the other direction:

While a battery is running, that worktree has no committable state. Not for the files the battery names, and in practice not at all, because git add -A does not know which those are.

What did not catch it, and what would have

./check.sh	started, killed at ten minutes by machine load (four review agents), deferred to landing time with a note saying so. The note was honest and the gate still did not run
the battery itself	ran and reported 9/9 killed — correctly. A battery mutates a committed target and restores it. It has no opinion about what else was committed while it worked
--check-anchors	green. The anchor was present; it was the arm that was gone
reading the commit	src/pinakes/template.py \| 2 +- sat in the stat output under a subject that promised a fragment. Nothing but a human reads that line

The cheap durable guard is the last row: a commit's stat is read before the commit is made, and a src/ path under a Retro: or Docs: subject is a stop. The expensive one is the first: a gate deferred because the machine is loaded is a gate that did not run, and the branch was pushed in between.

The generalisation, because this is the second instance of it this increment

The earlier fragment in this branch is about a mutant that died for a reason nobody read. This one is about a mutant that was never in the report at all. Both are the same shape: the mutation harness's output was believed about a state nobody looked at. 9/9 killed was true of the file mutate.py held. It said nothing about the file git held, and the two were different for seven seconds.

Three mutants killed for the wrong reason in one increment, and the pattern has three shapes (20260903 09:03)

A mutation battery reports KILLED when the named test fails. It does not report which assertion failed, in its summary table — and the table is what gets read, quoted into a commit message and carried into a fragment. Three rows in the S4 battery died on an assertion other than the one their name claimed. Each is a different mechanism, and each is generalisable.

1. The mutant was a safe version of the edit it names

Row: the refusal echoes the value it is refusing, straight to the terminal. It substituted {text!r} — and repr escapes a surrogate and an ESC by construction, so the mutant produced a terminal-safe echo. Measured:

assertion	{text!r}	U+…{ord} in {text}
"U+DCFF" in message	False ← the only kill	True
"\udcff" not in message	True	False
"\x1b" not in message	True	False

The kill came from the words U+DCFF going missing. The two assertions the test's docstring calls load-bearing — never echoes the value — were never measured at all, and would have survived a mutant that actually echoed.

The check: a mutant is written against a property, so name the assertion it must fail before writing it. "Does the test go red" is the weaker question, and it is the one the tool answers.

2. The mutant raised a different exception, so every assertion after raises was unreachable

Row: the surrogate arm is removed. Under it, the refusal never fires, Path.write_text raises UnicodeEncodeError, and pytest.raises(TemplateError) does not catch it. The test dies at the raises, so the line below it — assert not root.exists(), which the docstring calls the load-bearing half — never runs. Measured against the real init(): root.exists() True, pinakes.toml 0 bytes.

This is structural, not a slip: assertions written after a pytest.raises block are pinned only by mutants that keep the raise. A mutant that removes the raising code cannot reach them, however green the report looks.

No edit to template.py could fix it, because render_manifest runs before root.mkdir — so the row moved to src-pinakes-init.toml and mutates the ordering instead. It now kills on assert not True.

3. A more general arm below shadowed the specific one

_TOML_ESCAPES maps \b, \f, \n, \r to their single-letter escapes. Every one of them also satisfies the \uXXXX fallback two branches down. Drop any entry and the manifest still parses and still round-trips the exact value — every round-trip test in the file stays green, and no mutant could distinguish four lines of live code.

What separates them is only the bytes a human opens: name = "a\nb" against name = "a b". That is now asserted, in the same shape as the tab control.

The strongest demonstration was accidental. The planner, having just read the description above, tried to build the unescaped case by dropping \n from the map — and the output still came out escaped, because the fallback caught it silently. It had to disable both arms to see the defect. A shadowed line resists being shown to be shadowed, by someone who already knows it is.

The check: for any specific case, ask whether a more general arm below it produces an acceptable answer. If it does, the specific arm is unobservable and the test suite is silent about whether it exists.

What the tool does and does not print

tools/mutate.py does print the failing assertion, on the per-mutant line. All three of these were found by reading that line. What it does not do is put it in the summary table, which reads

| <row name> | KILLED | <test id> |

— the row's own claim about itself, and no evidence for it. That table is what gets read and quoted. Moving the reason into it is rowed, not built here.

(This section is itself the fourth claim in this increment asserted before being run: an earlier draft said the tool prints no kill reason at all. It was written inside the paragraph about evidence going unread.)

Why the corpus could not have found any of the three

Same shape as this increment's surrogate finding: every artefact asked does the test go red, and all three defects live in why it went red. A count cannot carry that, and neither can a green suite. What found them was a review pass that re-derived each kill's reason — one question per row, asked of evidence the tool had already printed.

The summary sentence disagreed with the table one screen above it (20260903 10:23)

MEDIUM — a plan's prose register and its own table register drifted apart, and the person who noticed was the one building from it, not anyone reading it.

plans/20260825_1240-run-pinakes-sweep.md carried, in its build-order discussion, the sentence "S1, S5, S6, S8, S9 all share one shape — an input the tool accepts and then mishandles, rather than refuses". Its own findings table, some four hundred lines earlier in the same file, defines S6 as a message that fires on ordinary deletion — no input is involved anywhere in it. The user performs a correct, complete deletion and the tool makes a false statement about what it did.

Three things are worth keeping out of it.

The generalisation was the defect, not a typo. Four of the five findings really did share that shape, which is exactly why the fifth was swept in without anyone re-reading it. A summary is a claim about a set, and a claim about a set is only as good as the member you checked last.

The document's two registers had already been recorded as a failure mode here — twice — and it still recurred inside the file doing the recording. 20260825_1252-plans-sweep-findings.md holds two registers of the same facts and twelve of its 27 rows stopped describing the tree; the rule written from that says where a dated snapshot and a ## Build order disagree, the build order wins. What this instance adds: the two registers can be prose and a table in one file, close enough that nobody thinks of them as two registers at all.

Building is a stronger read than reading. Nothing about the sentence looks wrong. It reads as a competent summary, and it survived every pass over this plan until a coder had to decide whether S6 belonged in the same guard as S8 and S9 — a question the sentence answers wrongly and a build answers correctly. The same pass returned two other things the plan did not have: a third arm of S8 (-k 0 is falsy, so limit or manifest.retrieval.final_k silently substitutes the default) and a third state of S6 under its decided fix. Cheapest available review of a planning document: give it to someone who has to act on it.

Corrected in place: the sentence is now a three-mechanism table — a mishandled input (S5, S8, S9, and S1 shipped in 0.32.0), a false reporting predicate (S6), and state that never clears (S7, which the original sentence never mentioned at all).

A decision that did not foresee its third state (20260903 10:24)

MEDIUM — an answered decision can still be short a case, and building it is what finds out. D-37 was answered 20260825: option E — gate the move hint on the orphaned sidecar, not the mint count. It is a clear instruction and it is right. Enumerating the states it applies to, before writing anything, turned up three where the decision reasons about two:

on disk after the change	orphaned sidecar	id minted	hint should fire
file and sidecar both deleted	no	no	no — this was the S6 false positive
file moved and edited, sidecar left	yes	yes	yes
only the file deleted, sidecar left	yes	no	yes, but nothing was minted

Option E fixes when the hint fires and is silent on what it says, so the third state passed the new gate and went on printing "so a new id was minted" — false. The wording is implementation, so it was mine to take, and the sentence now reports the state observed rather than the conclusion inferred. A decision that settles a predicate has not thereby settled the sentence the predicate prints.

The same shape, twice in one increment. S8 records -k -1 and -k -100. Enumerating instead of trusting the record found -k 0: the width is read as limit or manifest.retrieval.final_k, so a falsy 0 silently meant use the default — the user asks for nothing and receives ten passages. A third arm of a defect whose record listed two.

The transferable move is the table, not the finding. Both came from writing out every state the mechanism can be in and checking each against the rule, rather than checking the rule against the states someone had already written down. A recorded finding enumerates the cases its finder happened to try, and reads exactly like a complete list.

A fixture named for a scenario it did not build (20260903 10:24)

HIGH — a test can be green, correctly named, and modelling a different world. S6 — pnk sync announcing "moved without its sidecar" and "a new id was minted" on every ordinary deletion (quoted in halves: the joined sentence is now a retired row) — was covered by a test called test_rename_plus_edit_without_the_sidecar_is_reported_as_such. The name is accurate about the intent. The fixture passed () for the walk's sidecars.

A file moved without its sidecar leaves two halves on disk: the new path carrying no sidecar, and the old path's sidecar carrying no document. Passing no sidecars at all models something else entirely — a file moved and its sidecar deleted, which is indistinguishable from deleting one document and creating another. The test therefore asserted the hint fired in a state where firing is wrong, and it passed, because the code fired the hint on every vanished path. The fixture and the defect agreed with each other, and the agreement read as coverage.

The check that would have caught it is not "is this tested?" but "does the fixture build the state the name describes?" Two seconds of reading the scenario against the assertion, which is the reading nobody does on a green test.

The same increment showed the second half of it. Nothing anywhere asserted the sentence pnk sync prints for this case — the pairing tests pinned the predicate and stopped there, so the gate could have been fixed in pairing.py while sync.py went on printing the false sentence, and every test would have stayed green. A predicate and the words it produces are two surfaces, and pinning one is not pinning the other. Both are pinned now.

The rollback took the fix with it (20260903 10:24)

HIGH — a fix that unit-tests green and does nothing, because a transaction boundary ate it. S7's clearing was written where it reads best: at the top of _apply, once, before a path is re-attempted. Repair a document and the row goes. That much was true and the test proved it.

Then running the real thing three times against one broken document produced three rows. The failure paths in sync call connection.rollback() before recording, and the rollback discards the DELETE sitting uncommitted in the same transaction. The clear was written, then silently undone, then a fresh row appended — for every sync, forever. The half of the defect the fix was also supposed to cure survived it completely, and no unit test I had written could have shown that, because each one exercised a single sync.

What found it was running the command three times and counting rows — not reasoning about the code, which had already concluded the fix was complete. The repo's own rule says a seam removes an inch of the real path from coverage and that defects concentrate there; a transaction boundary is exactly such a seam, and it is invisible in the source line where the bug appears to live.

Two lessons, and the second is the sharper one. A write inside a block that may roll back is not a write. And "the test is green" answers a narrower question than "the defect is gone" — here they differed by an entire half of the finding.

A third came free, from the same habit. The first version cleared on every action except Skip, on the reasoning that a skip means nothing was attempted. Running it showed Skip carries two unrelated meanings: an unreadable document being held, and an ordinary unchanged one. They want opposite answers — and repairing a hand-edited sidecar back to its original bytes changes neither content nor sidecar hash, so the repair a user is most likely to perform arrives as "unchanged", and the fix missed precisely the case it was written for. Skip now says which it is. One name quietly serving two states costs nothing until something has to tell them apart.

A green battery inherits its author's framing (20260903 10:48)

MEDIUM — the mutants a fix's author can imagine are the failure modes that author already guarded against. Row 7's battery section ran 39 mutants and killed 38. Every one of the 38 was written by the person who had just written the fix, and every one died to a test that same person had just written. That is a closed loop, and it reads as proof.

The 39th is the one worth keeping. It widened S6's gate from document.path in orphaned_documents to the looser document.path in sidecar_by_document, and it survived — which looks like a missing assertion and was not one. Applied against the full suite it was green on 2398 tests: an equivalent mutant. The predicate I had introduced as the fix does not, at that program point, constrain anything the looser one does not.

Three things went wrong at once, and only the first is the kind a gate catches.

1. The mutant was equivalent. Not a finding.
2. Its kills selector named a test of a different function — _orphans, which never reaches that gate. tools/batteries/README.md says the selector is the half that historically rots; here it was wrong on the day it was written, and a SURVIVED row hid it, because a row that survives never exercises its selector.
3. The justification I wrote for withdrawing it was false. I wrote that control reaches that line only for a document being soft-deleted, so its path is necessarily absent from after.files. The conclusion is true. The reasoning assumes exactly what it needs to prove, and two routes do leave a present path unhandled and drop it into that loop. What actually closes the case is a three-step argument about claimed_by_id: both routes require the id to be claimed by a sidecar beside a present file, and every such sidecar is reached by one of two handled_ids.add(sidecar.id) sites before the vanished-path loop runs.

Number 3 was caught by a peer trying to refute the withdrawal, and failing. They built a present-path counterexample, ran it against both the real module and a mutated copy — anchor asserted to match once, __pycache__ cleared, pinakes.pairing.__file__ checked so that a null result could not have come from importing the unmutated file — and got an empty result from both. The refutation failed; the argument it was aimed at was replaced anyway, because attacking it is what exposed that it had never been load-bearing.

Lesson: a battery written by the author of the fix measures the author's imagination, not the code. Its green is evidence that the fix does what its author meant, never that what they meant was right. Two cheap things buy back most of the gap — run a surviving mutant against the whole suite before believing it is a finding, since an equivalent mutant and a missing assertion are byte-identical in the report; and have someone who did not write the fix try to refute the reasoning rather than re-read the diff, because the reasoning is where the author's framing is load-bearing and the diff is where it is invisible.

Second-order: a SURVIVED row never runs its own selector, so a battery's survivors are exactly the rows whose second half is unverified. The one place the report cannot check itself is the one place it asks to be believed.

See tools/batteries/src-pinakes-pairing.toml, which carries the withdrawn row's reasoning in place of the row.

A control must be shown able to fail (20260903 12:58)

HIGH — and it recurred an hour and a half after the same class was written down, by the same session, in this tree. Row 8's symlink guard reported a symlink to a real directory as "resolves to nothing … its target is missing or the link loops" — false on both counts, because is_file() is False for a directory target exactly as it is for a dangling one.

The part worth keeping is not the bug. It is that a control had been written against precisely this risk, and it could not fail. Its docstring said so out loud:

Without it, unresolvable could be populated by is_symlink() alone — which would report every aliased document in a KB that uses links deliberately, and sync.py already has a branch explaining that docs/alias -> docs/real is a supported shape.

The shape named there is a link to a directory. The shape the body built was alias.md -> elsewhere.md, a link to a file — which passes is_file() and therefore never reaches the is_symlink() branch at all. The assertion held for the correct guard and the broken one alike. Measured, not argued: with the fix reverted, the new directory-symlink test fails and the other six symlink tests stay green, which is the control demonstrating its own vacuity.

This is [[a-fixture-named-for-a-scenario-it-did-not-build]] again. That fragment is stamped 20260903 10:24 UTC; the commit carrying this instance is 6226b09, 20260903 12:02 UTC (both read with one instrument, TZ=UTC git log --date=format-local:, because format: renders in the committer's own offset and mixing the two silently compares different clocks). That fragment's rule — does the fixture build the state the name describes? — was already written, already in the tree, and did not transfer, because the second instance did not look like the first: this was a control, added deliberately against a named risk, in a test file that had just been reviewed. Believing a control is the failure mode; writing one is not enough.

The repository already owns the remedy in another register. tools/batteries/README.md requires killing one known-catchable mutant before believing a battery — "a run with no kills is a broken harness, not a clean bill." A control is the same instrument: it asserts that something does not happen, so nothing about its green tells you it was ever wired to the code. Revert the guard and watch the control go red, or it certifies nothing. That is the same test as "pinned by test X" is a claim about a failing test, applied to the negative case, where it is easier to skip because there is no fix to revert — you have to reach for the broken version on purpose.

And it happened a third time in the same increment, caught by the mutation pass rather than by reading. The battery row that shrinks the closed set — drop "pdf" from SOURCE_TYPES — was pointed at the control written to catch exactly that, test_every_valid_source_type_is_still_accepted. It SURVIVED. The control's loop is for source_type in SOURCE_TYPES:, so it iterates the constant the mutant shrinks: remove a member and the loop simply stops testing it.

The diagnosis matters more than the fix, and tools/batteries/README.md § Reading a SURVIVED row names both possibilities: the row is a claim about a pair, and either half can be wrong. Here the coverage was fine and the witness was wrong — with the mutant applied, tests/test_chunk.py goes red at line 745, because test_source_types_names_every_value_source_type_can_return derives the expected members from source_type() over real filenames and never reads the tuple. So the repair was the selector, not a new test.

Three instances, one shape: an assertion that draws its expectation from the thing it is checking can only ever agree with it. A control looping over the constant, a fixture built from the code's own notion of the state, a docstring describing intent beside code that does something else. None of them can fail, and all three read as coverage. The only instrument that found any of them was one that tried to break the code and watched what did not notice.

A docstring can claim a surface the code never reached (20260903 12:58)

HIGH — written as intent, committed as fact, and it was the only record of the gap. The closed-set constant added in row 8 carried this, in my own words:

That is what lets the CLI and the MCP server refuse one at the boundary instead of returning an empty result the user reads as an empty KB.

The guard was wired into pnk search's argparse and nowhere else. pinakes_search built its Filters from whatever the client sent, so source_type="markdwon" still returned zero passages under "nothing matched the filters" — the exact defect the increment existed to close, alive on the surface CLAUDE.md's naming table lists beside the CLI, under a sentence stating it was closed. eval.py's question loader had it too, where an unrefused typo scores that question zero recall on every run and reads off an eval table as a retrieval regression rather than as a typo.

The mechanism is ordinary and that is why it is worth a fragment. The docstring was written while the fix was being designed, when "the CLI and the MCP server" was a true description of the plan. The plan then shrank to the surface that was in front of me, and the sentence did not. No gate compares a docstring to the code it sits beside, and a reader has no way to tell a statement of intent from a statement of fact once both are committed.

Two things follow. First, a claim about another module belongs in a test, not a docstring — the check now lives in one helper the three surfaces call, so the sentence and the code cannot drift again. Second, when a fix names surfaces, enumerate the constructors rather than the surfaces: grep -rn 'Filters(' src tools is two seconds and finds every entry point, where recalling which surfaces exist finds the ones you were already thinking about.

A replace-all edits fixtures you never read (20260903 12:58)

MEDIUM — caught by a printed count, which is the only reason it was caught at all. Adding tests to tests/test_eval.py, I needed a kind: line in three new YAML fixtures and reached for a whole-file string replace. I printed the occurrence count out of habit rather than suspicion:

occurrences: 6

Three new fixtures, six matches. The other three were pre-existing fixtures belonging to other tests, and one of them was test_an_unknown_kind_is_refused, whose YAML came out carrying kind: lexical and kind: multihop — a duplicate key, where YAML silently takes the last.

Every other edit in the increment went through a helper that asserts the anchor matched exactly once and refuses otherwise. This one did not, because it was "just adding a line to my own new tests". The count was printed as information; had it been an assertion, the replace would not have run.

Two things I did not establish, and will not claim: whether the damaged fixtures' tests still passed — I restored the file before running them, so the honest answer is that I do not know — and whether git diff --stat would have shown it. What did show it was reading the hunks: git diff per-hunk, not the summary. A same-line-count replacement moves no totals, so a diffstat is weak evidence here either way.

The recovery is the part worth reusing: restore the file from HEAD and re-append only your own block, rather than inspecting and repairing what a replace-all touched. The block was still in memory as a string, the restore was one command, and the resulting diff — 53 insertions, zero deletions — is a checkable claim that nothing existing changed, which "I looked and it seemed fine" is not.

A vague sentence made specific, and false (20260903 12:58)

MEDIUM — the fix made the message more precise, which is normally the improvement, and that is how it got worse. The retrieval confidence reason used to say "nothing matched the filters" even when no filter had been passed. Row 8 split it, and the unfiltered arm became "this KB has no active documents to search".

That sentence is false whenever an active document produced zero chunks. _allowed_chunks joins chunks to documents, so an empty result means no active document produced a chunk, not no active document. A whitespace-only file syncs cleanly — chunk_document returns nothing for it, the row is written active regardless — and the user is then told their KB is empty while pnk doctor counts the document.

The old sentence was unhelpful; the new one was wrong, and confidently. An unhelpful message sends a user to look around. A specific false one sends them to look at the wrong thing, and it carries the authority of having been thought about. When a message is being made more specific, the question to ask is not is this clearer but what does the code actually know here — and the answer was: less than the sentence claimed, because the emptiness of a join is evidence about the join, not about either table.

The comment beside the first fix reasoned explicitly about soft-deleted documents and never considered the zero-chunk state, which is the shape of the error: the justification enumerated the states its author had thought of, and read as an enumeration of the states that exist. The probe now asks the documents table the question the join cannot answer.

The tag points at a commit CI never finished (20260903 13:18)

MEDIUM — a release tag can name a commit whose CI legs were cancelled, and nothing in the release procedure looks. v0.32.2 points at f23b602. That commit's CI run was cancelled 88 seconds later, when the next landing superseded it, and three of its jobs never completed: check (light), check (light pdf) and check (light pdf claude) — the three that actually run the test suite. The run's other eleven jobs succeeded, so gh run list shows it as cancelled rather than as anything alarming, and docs at the same sha is a clean green.

The release was already published by then. Nothing in docs/RELEASING.md reads CI at the release commit: the gate before the push is make release-check, which asks about the tag, and the verification after it asks about the artifact. Both are right and neither covers this.

What made it safe was measured, not assumed. The successor commit 058df49 merges that tree and its own CI run is green on all three check legs — checked per job, because a run's top-line conclusion says nothing about which legs ran. And the specific risk was nameable: this release moved src/pinakes/budget/prices.toml, and the last release that moved that file took both [pdf] legs red for 44 minutes. Locally those legs skip — pinakes[pdf] is not installed in the primary checkout — so a green ./check.sh on the release branch could not have seen it. 058df49 carries the re-stamped as_of and ran both legs green, which is what closes it.

Two things generalise.

A cancelled run is not evidence, and neither is a successor's green run. The successor tests a different tree — yours plus whatever superseded you. It is good evidence only when you have said what the difference is and why it cannot mask your defect. Here the difference is another session's row-8 work in cli, search, serve, sync, chunk and eval, and the leg at risk was a money trace in tests/test_pdf_trace.py that none of them touches.

The window that produces this is the same window the procedure deliberately uses. Landing and publishing are separate steps here, and docs/RELEASING.md § A fragment that arrives after the release commit but before the tag treats the gap as a feature. A peer landing inside that gap is therefore normal, and cancelling the release commit's CI is what a peer landing inside it does.

Lesson: after pushing a release tag, read the CI run at the tagged commit per job, not per run. If its check legs were cancelled, name the commit whose green run covers the same tree and say what differs between them — and say which leg the release's own diff put at risk, because a local ./check.sh skips whatever extras the machine lacks.

A gate that never ran the version it promised (20260903 13:50)

HIGH — requires-python = ">=3.13" had been a sentence in a manifest, not a tested claim, for the life of the project. Nothing in this repository pins a Python: no .python-version, no setup-python step, and uv sync --frozen resolves to the newest interpreter available. CI's matrix varies the extras — [light], [light,pdf], [light,pdf,claude] — and never the interpreter, so every leg ran 3.14. The floor was declared in one file and exercised in none.

What it cost: pnk sync crashed with a raw PermissionError on 3.13 for a symlink into an unreadable directory, and that crash is present in every published release tested — 0.32.2, 0.32.1, 0.30.0 and 0.25.0 all reproduce it under uvx --python 3.13.

The tell was visible and read as noise. The same commit produced a green gate and a red gate: ./check.sh passed in the worktree and failed in the primary checkout. That asymmetry has a note in private memory — worktrees get different Pythons; the primary checkout is the 3.13 outlier — and it was filed there as an inconvenience to route around rather than as the project's only instrument that ever touched the supported floor. The one machine that disagreed was the one machine testing what the manifest promised.

Two rules come out of it.

1. A version range in a manifest is a claim, and claims get gates. Every declared axis with a minimum — Python, an OS, a dependency floor — needs one leg that runs at the minimum, or the minimum is decoration. The new minimum-python job also asserts the interpreter it got (sys.version_info[:2] == (3, 13)) rather than trusting --python 3.13 to have been honoured, because a leg that silently ran 3.14 would be a job going green about the one thing it exists to check — the same failure this fragment is about, one level up.
2. When two environments disagree about the same commit, that is data, not friction. The instinct is to make the odd one match. The question to ask first is what is it testing that nothing else is, and only then whether to align it. Here, aligning would have destroyed the only signal in the system.

Related: [[pinakes-worktrees-get-different-pythons]], which recorded the divergence and read it as a hazard. It is that too. It was also the coverage.

A measurement with an unnamed population, copied forward three times (20260903 13:50)

HIGH — the sentence said measured, and it was. It was measured on one interpreter and written as though it were about the language. Row 8 left this comment in sync.py:

Measured, not assumed: exists() swallows the PermissionError and returns False.

That was true. It was run, the value came out of the run, and the phrase measured, not assumed was earned — on Python 3.14. On 3.13 Path.exists() raises, and pyproject.toml promises 3.13. The claim's population was one interpreter out of the two this project supports, and nothing in the sentence said so.

Then it was copied into two more places — a test docstring and a mutation-battery row comment — each time as settled fact, each time propagating a scope nobody had ever stated. Three locations, one measurement, zero mentions of what it was measured on.

The failure mode is not laziness, it is that the guard against it was already engaged. Measured, not assumed is the phrase this repository uses to mark a claim as checked, so the sentence arrived pre-defended: a reader looking for unverified assertions skips it, and a reader looking for scope sees a word that promises rigour. A false claim with no evidence gets challenged. A true claim over an unnamed domain does not, because the challenge it invites — did you check? — has already been answered.

The question that catches it is not "did you measure?" but "measured on what?" For anything whose answer can differ across an axis the project spans — interpreter, OS, filesystem, extra, locale — the population belongs in the sentence. The corrected comment now carries a table with both interpreters named and both versions given, because a table cannot omit its own columns the way a sentence can omit its own scope.

Related: [[a-null-result-carries-no-information]] and the 20260831 record of six wrong claims in one day, each a valid inference over a population nobody named. This is that shape again, with the extra twist that the inference here was not merely valid but genuinely executed.

I almost reported an invariant break I had not measured (20260903 13:50)

MEDIUM, and the outcome was fine — which is the reason to write it down. Mid-fix, one test failed on 3.14 but not 3.13. The inference came immediately and it was a good one: at the pairing site, is_file() returns False on 3.14 instead of raising, so the document's existing sidecar goes unread, so pairing sees no sidecar, so it mints a fresh ULID for a document that already has one — ULID permanence, in docs/INVARIANTS.md, broken silently and per-interpreter.

Every step of that chain was plausible. The mechanism was real, the failing test was real, and the conclusion was the most serious class of defect this project recognises. I had a message half drafted to the planner saying so.

I ran it instead. There was no duplicate ULID. 3.14 refuses the document too — via SidecarError rather than PermissionError. The real divergence was the error class and its remedy, a genuine defect and a much smaller one.

The lesson is not "measure before asserting", which was already the rule. It is when the rule is hardest to follow. The pull to skip the measurement was strongest precisely because the hypothesis was alarming: an invariant break is urgent, a peer was waiting on an answer, and the reasoning felt like it had already done the work. Severity creates the impression that speed is the responsible choice, and a chain of correct-looking steps feels like evidence because each link is checkable — but the conclusion is only ever as good as the run nobody did.

So: the more serious the claim, the more it earns a measurement rather than exempting itself from one. The escalation-worthy finding is the one to check first, not the one to send first. A concrete marker for next time — I was about to write the words "silently mints a second id", a statement about runtime behaviour, without having observed any runtime behaviour. A sentence describing what the code does at run time and sourced only from reading it is a hypothesis wearing a report's clothes.

Related: [[the-command-ran-the-number-was-typed]] — there, the measurement happened and the figure did not come from it. Here the figure would not have come from anything at all.

The fix stopped at the boundary I reasoned to, not the one the defect had (20260903 14:10)

What happened. The crash arrived from pnk sync, so I scoped the fix to the source walk in sync.py and argued the scope was right: that is where a corpus path gets stat-ed, and the other 41 is_file()/exists() calls in src/ are about paths Pinakes itself created under .pinakes/. The argument was sound. The scope was wrong. My own adversarial pass found doctor.py failing on the same state, linkscan.py returning a wrong answer on it, and — worst of the three — an orphan check that offered --prune for a document sitting on disk, which would have destroyed a permanent ULID. None of that was visible from where the crash was reported.

Why. I derived the boundary from the symptom's call path instead of enumerating the condition's reach. The condition is "a corpus path this process cannot stat", and the way to find its reach is to list every place a corpus path is stat-ed and ask each one what it does with False. That is a grep and a read, not a deduction — and it is cheaper than the deduction was. Reasoning from one entry point produces a boundary shaped like the entry point.

What it cost, and what it nearly cost. Three commits instead of one, which is nothing. But the --prune bug is interpreter-independent and predates this branch: it was reachable on every supported Python, and a fix that had stopped where my reasoning stopped would have shipped a version-independence claim while leaving a data-destroying prompt in place one file away. The review found it; the reasoning had excluded the file it was in.

The rule. When a fix is for a condition rather than a line, establish the scope by enumeration and say so — grep the predicate, read each site, record what each does when the answer is False. Then state the boundary as a set you looked at, not as a category you inferred. A category is a claim about files you did not open.

And check what False means at each site before reusing the helper. Two of the four sites wanted the new version-independent is_regular_file; the orphan check wanted os.path.lexists, because there False means nothing is here and it had been reading it as nothing readable is here. A shared helper makes a class of bug go away and a class of bug easy to write: the sites that must distinguish absent from unreachable are exactly the ones a sweep will quietly get wrong.

A two-way selector that can only fall one way (20260903 15:23)

What happened. Verifying the published 0.32.3 fix for the --prune bug, the check was grep -qi 'orphan' && echo "ORPHAN REPORTED" || echo "the id is safe". On the fixed wheel pnk doctor printed OK orphaned sidecars: none. The word orphaned is in that line, so the selector matched, and the harness printed ** ORPHAN REPORTED — a live document's id is offered to --prune ** about a release that had just been shown correct — with the output it had misread printed two lines above the verdict.

Why it is not a typo. The selector had two branches and only one of them could ever be reached by evidence. grep -q matching is a claim; grep -q not matching is the absence of a claim, and the || branch spends it as though it were the opposite claim. So every failure mode of the instrument — a wrong pattern, a command that died early, an empty string, a renamed message — lands in the same branch as a genuine pass. &&/|| on a matcher is a one-way test wearing the shape of a two-way one, and it is the shape most of my quick verification snippets have.

Why nothing caught it. The check ran, printed, and exited 0. There is no gate for the measurement was performed and the number was typed anyway, because the checking step is present. It was caught only because the surrounding two lines of raw output happened to be printed beside the verdict, and a human-shaped read of them disagreed. Had the harness been tidier — verdict only, no evidence — it would have shipped a false negative about the most severe bug of the day, in the direction that says still broken, which is the direction that gets acted on.

The fix, and it generalises past this check. State both expected forms and refuse to guess when neither appears:

if   printf '%s' "$out" | grep -qE 'WARN +orphaned sidecars: [1-9]'; then  verdict=BROKEN
elif printf '%s' "$out" | grep -qE 'OK +orphaned sidecars: none';    then  verdict=FIXED
else                                                                       verdict=INCONCLUSIVE
fi

Three outcomes, not two. INCONCLUSIVE is the one that carries the information the || branch was destroying: the instrument did not recognise what it was looking at. It is also the branch that fires when someone rewords the message, which is when a silent selector is most likely to be wrong and least likely to be doubted.

The rule. A selector deciding between two states must be able to see both. Write the positive pattern and the negative pattern, and make the fall-through say so — never match && A || B. This sits directly beside two rules already written down here: a null result carries no information until the selector is shown able to fire, and a claim resting on an instrument you chose must state the selector. This is the third face of the same coin, and the cheapest to check: read your own snippet and ask which branch a broken instrument lands in. If the answer is "the good one", the snippet cannot fail visibly.

Two more instrument failures in the same twenty minutes, both of which printed a clean-looking pass. A hand-written pinakes.toml was refused for a missing key, so the crash probe never reached the crash. And with the embedding backend absent, pnk sync exited before the walk it was built to exercise, printing no PermissionError having tested nothing — the same class as a fixture whose sidecar is missing, so the loop that raises is never entered. Three in one session says the failure mode is the harness, not the day: every one of them stopped short of the code under test and reported the absence of a symptom as the absence of the defect.

A fixture that never reached the code under test (20260903 16:10)

What happened. A peer handed over a reproduction for row 31's first shape: make the [sources] root a symlink into a blocked ancestor — kb/docs -> ../blocked/realdocs, with blocked at 0o000 — and observe that Path.is_dir() raises on 3.13 while returning False on 3.14. It came with the interpreter divergence measured on both versions, and with a named source line. Rebuilding it as a fixture, it never ran: manifest._check_include_containment refuses a root that resolves outside the KB, so the manifest fails to load and the walk is never reached at all.

What was wrong, and what was not. The interpreter divergence was real. What did not survive was everything downstream of the fixture: the reachable shape needs the symlink target inside the KB (kb/docs -> store/realdocs, kb/store at 0o000), and with that shape the 3.13 raise is at is_dir() and not, as reported, one line earlier at .resolve() — resolve() is non-strict here and does not stat. A precise, correctly-measured claim about the wrong tree.

Why a peer's reproduction is the easy one to accept. It arrives already carrying the things that normally warrant trust: a measurement, both interpreters, a file and a line number. None of those is evidence about whether the state it builds is reachable by the code under test, and that is the one property a reproduction has to have. A fixture that errors before the code under test can still print a plausible failure — a different failure — and the reading of it is confirmatory either way.

The check that catches it, and it is one line. Assert something the fixture must have reached before believing what it reports. Here that is the state right after the setup: the index contained store/realdocs/a.md, which is only true if the manifest loaded and the walk ran and keyed the documents by the resolved root. The test now carries that assertion with a message explaining what it is for, rather than beginning at the failure the fixture is meant to produce. Opening a reproduction with a proof that it got as far as the code under test costs one assertion and is the difference between a regression test and a fixture that would pass on an empty function.

What was right about the exchange. The peer had already applied this to itself once that day — correcting a claim of its own that was true of its fixture and false of the language — and said so in the handover. That is what made checking the fixture the obvious next step rather than a slight. The rule a peer's claim is not evidence until you have checked it survived being handed a claim that was 80% correct, which is the hard case: the wrong 20% was load-bearing, and every signal of rigour attached to the 80%.

The positive counterpart, done for this increment rather than assumed. The rule "unit tests prove a component honours a parameter — only running proves the parameter arrives" is the same statement from the other side, so the fix was exercised the way a user meets it before landing: a real pnk sync over a 30-document KB with docs/vault at chmod 000. It printed failed: docs/vault: directory could not be entered: Permission denied. with the remedy, 0 removed and 30 unchanged, and exited 1; pnk doctor then exited 0 with nothing to say about orphans or retired rows, and pnk search "appraisal criteria" answered from the held document inside the locked directory. Every one of those is asserted by a test as well. The run is what shows the assertions are about the command the user types.

A hook that could not fire for half the class (20260903 16:10)

What happened. The decided fix for row 31 was ask the filesystem instead of inferring: replace a silent root.glob(pattern) with a walk carrying an os.walk error hook, collect the directories it could not read, and hold the documents under them. The decision named two fixtures — an unreadable root and an unreadable subdirectory — and both were measured on both interpreters before it was taken. It was still insufficient, and the gap was not in the reasoning.

The measurement that found it. Before writing anything I enumerated the modes, not the cases: 0o000, 0o100, 0o400, 0o500 on a subdirectory, asking each of root.glob("**/*.md"), os.walk(onerror=…) and os.path.isfile what it said. Four rows, one script. The 0o400 row is the one nobody had:

mode	glob yields	onerror fires	os.path.isfile
0o000	nothing	yes	False
0o100	nothing	yes	True
0o400	the entry	no	False
0o500	the entry	no	True

At 0o400 the directory lists and cannot be entered. scandir succeeds, so no error is raised for any hook to receive; the glob hands back the name; and every stat on that name then fails one at a time. So the decided mechanism — an error hook — is structurally blind to it, and the walk was still deleting the documents underneath at exit 0. On Python 3.13 it did not even do that quietly: Path.is_symlink() lstats, lstat needs +x on the parent, and the next line raised.

Why the decision could not have found it. Both named fixtures make the directory unlistable, and unlistable is the only state that fires a hook. A decision reasoned from two instances of one state will specify an instrument that covers exactly that state, and the specification then reads as complete because every fixture in it passes. The gap is invisible from inside the case list; it is only visible from the axis the cases are points on.

The generalisable move. When a fix is specified in terms of an instrument — a hook, a probe, a callback — enumerate the states of the thing being instrumented and ask the instrument about each one, before writing the fix. Not more fixtures of the same shape: the axis. Here the axis was four permission bits and cost one script; the two cases in the decision were both the same point on it. The question that finds this is "what can this instrument not see?", and it has to be asked while the instrument is still a choice.

What it changed in the build. The fix is two halves rather than one, and they are not alternatives: paths.unreadable_directories collects what the hook can see, and paths.unreachable catches the rest one candidate at a time, with the walk recording that candidate's parent — always the culprit, because a glob cannot descend past an untraversable directory, so every entry it still yields from one is a direct child. A test pins the limit itself (test_unreadable_directories_cannot_see_a_directory_that_lists_but_cannot_be_entered), because the per-candidate half looks redundant beside the collector and the next reader will delete it otherwise. A stated limit needs a test as much as a behaviour does — it is the only thing that turns red when someone acts on the belief that one half is enough.

A postscript, because it is the same lesson pointed at the test harness. The mutation battery for this work reads 57 of 57 killed on 3.13, and 56 of 57 on 3.14 — one survivor, the row that reverts doctor's directory guard to the pathlib spelling. That row is not a gap. Path.is_dir() raises on 3.13 and returns False on 3.14, so on the newer interpreter the mutant merely skips the root and the test's assertions still hold. The row's own comment said so before the run, which is the only reason the survivor is readable as a prediction rather than as a hole; written afterwards it would be indistinguishable from an excuse. An instrument that cannot see a case has to say so in advance — which is exactly what the 0o400 row above is about, one layer down.

Row 32 — a ruling that was wrong on a premise, and a fake that outlived its branch (20260904 02:02)

HIGH — a decision can be wrong because of a fact, and the fix is to measure the decision, not to re-argue it. The plan ruled paths.unreachable as the discriminator for both of this increment's fixes, and stated that it would "restore 3.14 to what 3.13 already does". It would not. unreachable is lstat-based, so on a symlink pointing into an unreadable directory it answers False on both interpreters. Building the ruling as written would have left 3.14 broken and regressed 3.13, moving that document from paid extraction unreadable into neither list — silently dropping a row 3.13 reports today, on the very shape that produced the ULID-deletion bug. The obvious repair over-fires in the other direction: a naive os.stat sibling answers True for a symlink loop, where 3.13 answers False. Neither candidate alone is parity. What settled it was a table of seven shapes on two pinned interpreters, with the controls — absent, dangling, loop, ordinary file — in the same run as the cases. The ruling was re-ruled within the hour on that evidence. The cost of measuring first was about twenty minutes; the cost of not doing it was a regression on the interpreter this project declares as its floor.

HIGH — a fake pinned to an implementation detail fails loudly when the detail moves, and goes silent when the detail's behaviour moves. test_an_unreadable_directory_is_refused_rather_than_ crashing monkeypatched Path.is_file to raise, and passed on both interpreters — while on 3.14 the real Path.is_file() raises for neither errno the test's own comment names, so the production branch it certified was dead. The test was measuring its own fixture. When the fix moved the call to paths.unreachable_through_links, the same test went red immediately, because the fake no longer intercepted anything. Both failures are the same seam; only the noise differs, and the quiet one had been shipping. The repair was to move the fake down to os.stat, the OS boundary, and to add a real chmod test beside it — the injected one now earns its place on the errno no permission fixture can produce and on running as root, where the chmod fixture skips.

MEDIUM — "the library does X" is often a measurement of one interpreter, written up as a fact about the library. The docstring being fixed here claimed pathlib "ignores ENOENT, ENOTDIR, EBADF and ELOOP … but nothing else". True on 3.13, and on 3.14 Path.is_file() swallows everything, EACCES and ENAMETOOLONG included. That single sentence is why an except OSError clause sat dead in production and why a test sat green on top of it. The replacement constant is named for what 3.13 does, not for what pathlib is believed to do, and its four members were measured on both interpreters rather than read out of CPython — with the one member whose fixture is not portable marked as measured-but-not-pinned instead of quietly asserted.

MEDIUM — an audit's list decays, and one of these had already been fixed by a neighbouring branch. Of the survivors this row carried, doctor.py:411 was cured in 514fe46 — row 31's own commit, in passing — while the row still described it as live and named the release that had walked past it. Checking git log -S on the exact line took a minute and removed an item from the queue. An unverified finding ages in both directions: it can be worse than raised, as the link one was, or already gone.

A gate I read but did not obey (20260904 02:12)

HIGH — I landed over a red ./check.sh, and the commit message says it was green.

The chain was ./check.sh > log 2>&1; echo "CHECK=$?" … git add -A && git commit. A ; where an && belonged. The exit status was captured, printed as CHECK=1, and then ignored by every command after it. f857e39 reached origin/main carrying the sentence "check.sh green, make docs green", which was false when it was written.

This repository already has the rule, in CLAUDE.md, and I have been quoting it at other sessions all day: a gate is only a gate when its exit status is what the next command reads. The canonical failure it names is check | tail && git commit, where tail's success masks the checker's failure. Mine is one step cruder — I did not mask the status, I printed it and carried on. Having the number on screen is not the same as branching on it, and a chain that reports a failure it does not act on reads exactly like one that passed.

The failure itself was environmental, and that is the part that makes this worth writing. test_a_target_outside_the_repository_is_refused returned -15 — SIGTERM — where it asserts 1. The subprocess was killed, not wrong. Another session was running a full suite and a mutation battery on the same machine at the same time. Re-run on main: that test passes, its whole file passes 70/70, and ./check.sh is green at 2452 passed. So the tree was never broken — which is precisely why the process failure is worth recording rather than quietly fixed. A red gate that turns out to be noise is the cheapest possible lesson; the same ; on a real failure lands the defect.

Two things to carry.

A signal-killed test is not a failing test, and the report cannot tell you which. assert -15 == 1 reads as an assertion failure. Only the value says otherwise, and only if you know that a negative return code is a signal. Under concurrent load on a shared machine this will recur, so read the number before diagnosing the code.

Verify before excusing. The environmental explanation is the convenient one, and this project's own record is full of convenient explanations that were wrong. It was checked three ways — the single test, its whole file, and a full ./check.sh on main — before being written down as noise. The order matters: if any of those had been red, this fragment would have been a defect report instead.

Lesson: ; between a gate and a commit is the same defect as piping the gate through tail, and it is harder to see because the failure is printed rather than hidden. Chain with &&, or read the log before the commit — never both a printed status and an unconditional next step.

Row 38 — a test that proved nothing, and a contract "verified" on one interpreter (20260904 02:55)

HIGH — a test can stop testing anything without ever going red, and the only way to know is to disable its instrument and watch it still pass. test_an_unreadable_linked_kb_path_is_a_warning_not_a_traceback monkeypatched Path.is_file to raise, and asserted that pnk doctor reported a WARN. Its fixture built the partner directory without a pinakes.toml — so the WARN it asserted came from the partner genuinely not being a KB, and the injected PermissionError decided nothing at all. I did not infer this: I disabled the monkeypatch line and re-ran, and the test passed unchanged. It had been green for the wrong reason, and no gate anywhere can see that, because a vacuous test and a sound one are the same colour. The replacement uses a real, synced partner KB behind a real chmod, so the permission is the only thing that can make it fail.

HIGH — "verified" in a docstring is a claim about the machine it was run on. why_not_a_kb's docstring argued that it may raise and that its callers guard it, naming exists(), is_symlink() and is_dir() as raising on an unreadable parent, and marked that verified. It was verified on one interpreter. On 3.14 those calls return False, so nothing raised, no except OSError fired, and three commands printed "no such directory" about a partner on disk. The docstring did not merely go stale — it was load-bearing, because three call sites carried comments justifying their guards by pointing at it. One sentence measured once propagated into three modules.

MEDIUM — the argument against a total contract was the thing worth re-reading, not the code. That docstring gave an explicit reason it could not be total: there is no answer it could return for "I could not tell" that a caller would not have to branch on anyway. That was false the whole time, and its own callers proved it — every one of them was already printing exc.strerror, which is exactly the answer. The same module argues the general case twelve hundred lines up, in resolve_path: a function three call sites each had to remember to guard is a function with the wrong contract. The counter-argument to a rule was sitting in the same file as the rule.

MEDIUM — the same seam failed for the fifth time in one increment. Five tests faked Path.is_file because that was the spelling the production code happened to use. Each went red the moment the call moved to a version-independent one — which is the loud failure. The quiet one had already happened: on 3.14 the real call raises for neither errno these tests name, so the branch each certified was dead while the fake entered it and they passed on both interpreters. Fake the OS boundary, not the caller's spelling of it.

Row 40 — a count whose meaning depended on something nobody recorded (20260904 03:14)

HIGH — an equivalent mutant and an unpinned assertion are byte-identical in the report, and I read one as the other. A committed row came back SURVIVED and I reported it as an unkilled mutant and as a coverage gap. Both halves were wrong. On the newer interpreter the mutated and unmutated programs behave identically — both spellings of the predicate answer False — so nothing could kill it, and the silence I called a gap is produced by the fixed code too, which makes it a different open question rather than a hole in this battery. What I actually had was narrower and still worth the row: the same battery gives two different counts on two interpreters, and the report never said which one you were reading.

HIGH — the fix asked for was the wrong fix, and building it literally would have been worse than building nothing. The row said to print sys.version. mutate.py's own sys.version is the launcher's, and the documented invocation is python3 tools/mutate.py — the system interpreter — while the tests run under uv run --frozen pytest, the project venv. Measured by making them differ on purpose: launcher 3.14.7, venv 3.13.15. The literal implementation would have printed 3.14.7 beside counts produced entirely by 3.13.15 — an answer that reads as measured and names a Python no test touched. A wrong number in a sentence that looks like evidence is worse than an absent one, which is why the unprobeable case says could not identify instead of falling back.

MEDIUM — placement is delivery. The pytest = … command has been printed in this tool's header the whole time and never answered this question, because the header is what gets left in the terminal and the counts are what gets pasted into a commit message. The new line goes beside the counts, and a mutant pins that ordering — a correct sentence in the wrong place is indistinguishable from an absent one at the moment somebody needs it.

MEDIUM — the honest way to report an instrument you cannot read. Two command shapes are probed because two are documented; anything else returns nothing rather than a guess. That is the stance this tool already takes towards a JUnit report it cannot parse — the outcome is unknown rather than SURVIVED — and applying it to the interpreter as well cost one if and removed a whole class of confidently wrong report.

Row 39 — I tested the copy that did not contain the change, and broke main doing it (20260904 08:30)

HIGH — a smoke test run against the wrong binary reads exactly like a passing one. I had just written the guard that refuses to land an ungated tree, and I wanted to watch it refuse. So I committed it as WIP and ran python3 tools/land.py <my branch> from the primary checkout, expecting a refusal. It landed, and pushed — because land.py executes from the primary checkout, which still held main's ungated copy. I had written the sentence "the gate runs in the branch's worktree and this refusal runs here" into that same file minutes earlier. The instrument did not contain the thing under test, which is the identical shape as the vacuous test found hours before: an injection that no longer intercepted anything, and a result that read as informative because nothing about the output says which code produced it.

HIGH — the change broke main, and the way it broke is the argument for the change. Six tests/test_land.py tests went red immediately: their scratch repositories never run ./check.sh, so the new guard correctly refused every landing they assert. The tests were right, the guard was right, and they had never been introduced to each other. What actually put a red tree on origin/main was not the guard — it was landing without running ./check.sh at all, which is precisely what the guard makes impossible. It could not stop me because it was not yet on main: this guard's own landing is the one landing it cannot police. That bootstrapping hole is worth stating rather than smiling at, because it is the general case — a gate can never cover the commit that introduces it, so that commit needs the discipline the gate is replacing.

HIGH — the planner caught a hole that would have made the guard blind exactly where it matters, and the measurement was worse than the report. The first design keyed the marker to the branch's tree. But git merge --no-ff combines the branch with a main that may have moved, and the result is then neither side's tree. Reported as one merge in three; measured across all four merges available, it was two of four — and both were mine. So the guard would have passed while a tree nobody had gated reached origin/main: the a clean auto-merge is not a correct merge case, with the guard asleep in the one situation it exists for. git merge-tree --write-tree computes the merge result without performing it, touching neither the working tree nor HEAD, and predicted the real tree in all four — so the check happens before the merge rather than after it with a reset to undo.

MEDIUM — the absence of an escape hatch is a feature, so it is asserted rather than assumed. A flaky or environmental red now blocks a landing; a suite killed by machine contention did exactly that the same night, correctly. The first person to meet that at 3am will want --no-gate, and a test now fails if one appears. The reason it does not exist is that the rule being skippable is what put the guard here in the first place.

Row 41 — four instruments for one audit, and the fix was already written down (20260904 08:54)

The audit itself was one afternoon's work. Getting an instrument that could be trusted took four attempts, and every failure was the audit's own subject turned on itself: a measuring device that did not contain what it claimed to measure.

HIGH — the decisive bug was a rule this repository already states, in a context I did not recognise as the same machine. CLAUDE.md tells a mutation harness to clear the module's __pycache__ after writing and after restoring, because CPython invalidates bytecode on (mtime-to-the-second, size). This audit edits a test file, runs pytest, restores, and repeats — many times a second — which is precisely that shape. I did not apply the rule, because I was building "an audit" rather than "a mutation battery". Two of fifteen sites then returned a different verdict on repeat, and one site was reported vacuous by a batch run while three separate single-site runs called it sound. Clearing __pycache__ took the instability to zero of fifteen. A rule filed under the wrong noun is a rule you will break.

HIGH — "still passes with its instrument disabled" is three findings wearing one label, and only one of them is a defect. The probe cannot, by itself, tell apart: (a) a test whose assertions do not depend on the injected condition at all; (b) a test where the real environment produces that condition anyway, so the fake is redundant; (c) a mis-attributed site, where the probe disabled an injection and then ran a test that never used it. All three print the same word. Case (c) was my first instrument attributing injections inside helpers to whatever test happened to sit above them. Case (b) is real and stayed: a 300-character filename raises ENAMETOOLONG on this machine for real — NAME_MAX is 255, measured — so that injection decides nothing here, while the test's own comment records the opposite on CI. That site cannot be ruled from this machine at all, and saying so is the result; picking either answer would have been an invention.

MEDIUM — the count moved 5 → 3 → 2 across instruments, and only the last one is reportable. Every intermediate number was a true statement about a broken measurement. The discipline that saved it was cheap and mechanical: probe every site twice and report only verdicts that agree, then cross-check the flagged ones by hand. The one genuine finding survived all four instruments.

MEDIUM — what the audit found, stated against its own trigger rather than against a preference. Fifteen injection sites on pathlib/os/paths predicates; one genuinely vacuous test, now fixed. The trigger for writing a general rule was more than one further vacuous test. One is not more than one, so no rule was written — deliberately, because a rule invented from a single instance is the ceremony this repository keeps warning about, and because the honest reading of a boundary result is to report the boundary rather than round toward the outcome that produces a deliverable.

Row 43 — I measured the function and made a claim about the command (20260904 09:38)

HIGH — an isolated measurement is evidence about what you called, never about what the user runs, and the gap between those two is where I put a false finding. Row 36 sent me to probe the pathlib predicate sites nobody had looked at. I called two of them directly, watched PermissionError on 3.13 and silence on 3.14, and reported two live defects on the declared floor — including "ends pnk sync in a raw traceback". The divergence was real and reproducible. The consequence was invented. Measured afterwards, against the real command: pnk sync on that state produces byte-identical reports on both interpreters, both call sites of the first function sit inside except (PinakesError, OSError), and the second is unreachable because manifest.load refuses an untraversable root before apply ever runs. Reachable count: zero. The planner filed a row and ranked it above a gate design on that report.

HIGH — the thing that caught it was the test failing to fail. I wrote the end-to-end test first. It passed. Then I reverted the one-word fix and ran it again — it passed again. A test green with and without the change under test pins nothing, and that is what sent me looking for the call path instead of banking the finding. That is row 41's own method, turned on row 41's author about an hour after it was written up, and it is the second time in one day the same check has caught something: once in the audit, once in the auditor.

MEDIUM — the strongest argument for closing a class rather than fixing instances came from being wrong. The reason my sync.py finding is unreachable is row 31's own guard: unreadable_directories refuses the directory at the walk, so the per-file path is never entered. A fix that landed the same morning, one level above the site, had already closed the class the site belongs to. Nobody planned that overlap and it is the clearest evidence for the ordering the queue keeps arguing about.

MEDIUM — what a hygiene change has to say about itself to survive. Both lines still changed — one word each — because the codebase has decided paths.py owns this question and these were the last holdouts on user paths. But each docstring now states plainly that no production path reaches it today, how that was measured, and what would change it: the walk guard narrowing, or a caller dropping its except. Without that last clause the next reader deletes the change as pointless, and without the first two they inherit my false claim in a comment.

MEDIUM — a test that discriminates on one interpreter should say so in its own docstring. Both new tests are red on 3.13 without their fix and green on 3.14 either way, because the unfixed spelling already returned False there. Stated in the tests rather than left for someone to discover when a 3.14-only run reports them as covering something.

Row 42 — six ways an instrument reports success, five of them mine (20260904 10:28)

Scope, first, because it decides what the rest is worth. Everything below was measured in this session, after a context clear at 09:54 UTC. I have not re-derived the row 41 audit, the floor audit's 18 → 8 → 5 chain, or the six-mutant episode, all of which happened in this session's previous context. A successor asked to write those up refused on the same grounds and that refusal was recorded as correct; repeating it here rather than reconstructing them from a peer's summary. The evidence for those lives in plans/20260901_1148-clear-the-user-facing-list.md row 32, in the retrospective fragments already spliced into this document, and — measured independently by the planner during this session, under its own name rather than on my testimony — in plans/20260904_1027-development-paused-handover-and-evidence.md, which is the document a resuming session should read first.

HIGH — the instrument added itself to the population it was measuring. The audit greps every tests/test_*.py for monkeypatch.setattr(Path|os|paths, …) on a line ending in ). A test file about injections writes exactly that as fixture data, so mine joined the set: 15 sites became 17, and probing a "site" that was a string literal edited a line inside a string, broke the file it lived in, and returned INCONCLUSIVE — the whole run went red over a site that does not exist. tests/test_mutate.py already carries a warning about the identical trap against test_verification.py's ^def (\w+) scan. The trap was documented, in this repository, for a different gate, and I walked into it anyway. A rule filed under the name of one instrument is a rule you will break with the next one — which is the same sentence row 41's own retrospective wrote about __pycache__, one tool over and one day earlier.

HIGH — my first mutant survived, and it was the mutant that was wrong. Testing the guard above, I wrote a poisoning line that matched the regex but did not end in ), which sites() also requires. It changed nothing, the test stayed green, and "the guard is pinned" was one keystroke away from being recorded. A mutant that cannot kill certifies the test, not the code, and the only thing that caught it was noticing that a guard I had just watched fail by hand had not failed under its own mutant.

HIGH — a report that named the environment the run finished in. environment() was called after the probe loop. At 10:23 the audit reported probed under Python 3.13.15; at 10:24, in the same directory with no uv sync between them, the next command answered 3.14.7. It is now read before the loop and again after, the report names the before value, and a run whose environment moved is UNATTRIBUTABLE. The mechanism was not established — the flip did not reproduce under a controlled sequence of uv sync --python 3.13, uv run, a pytest run and a clear_pycache — so this is a guard against an observed instability, not a fix for a diagnosed one, and both the code comment and this paragraph say so rather than inventing a cause.

MEDIUM — a comment that stated the intent its code did not implement. At the classification site: "A skipped test also exits 0, and a skip is not a pass" — followed by a check for no tests ran and 0 passed only. pytest prints 1 skipped, which matches neither, so an all-skipped selector fell through to VACUOUS with the exit status agreeing: a false finding, in the one direction that matters, because a sound verdict withholds a finding while a VACUOUS one is what a person then acts on by deleting a fake. The honest half of the scope: no owning test is geteuid-guarded today — those guards sit on their non-injecting neighbours — so this was a latent misclassification and not a live wrong verdict. It is the same shape as the why_not_a_kb docstring that asserted the opposite of its code: prose next to code is not evidence about the code, and it is read as if it were.

MEDIUM — an empty result set that certified a clean one. With no sites collected the report read 0 sites · 0 vacuous · 0 not ruled and exited 0. A reformat wrapping one monkeypatch.setattr across two lines is enough to produce it. Now a refusal, with the floor asserted by value — --min-sites 1 restores the hole in full, which is wheel_import_gate.py's own --min-modules 1 lesson arriving one tool later.

MEDIUM — I left a fake in the process-global subprocess module. module.subprocess.run = spy does not patch the tool's view of subprocess; module.subprocess is the module, so three tests replaced subprocess.run for the whole session and never restored it. They passed because nothing after them in that file shells out. In a file whose entire subject is instruments that quietly stop measuring, this is the joke writing itself.

MEDIUM — three timestamps I composed instead of reading, in a branch about measurement. A test docstring claimed 20260904 10:57 UTC while the clock said 10:23 — a measurement dated in the future. Two commit messages claimed 10:12/10:22 and 10:31 for runs whose own log artefacts say 10:01, 10:03 and ~10:05. All three were plausible, none was read. The rule read the clock, never compose it is in CLAUDE.md twice; what defeats it is that a wrong time never looks wrong. The fix that worked was mechanical: take the time off the artefact the run left behind (TZ=UTC stat on the log), never off memory of when it happened.

A note on instruments, since two of mine were wrong today. Reading the workflow journal I looked for a key named value; the key is result. My reader printed None for every agent and I briefly believed two design agents had died — they had returned complete answers. A null result carries no information until the reader is shown able to fire, and I had not shown it. The other: TZ=UTC does not make git log --date=format: render UTC, and format-local renders in the machine's zone — here IST, two hours off. Every time I quote below came from TZ=UTC plus --date=format-local, verified against date -u in the same command.

What the ownership boundary actually cost, measured rather than recalled. It bound twice today. Once for docs/VERIFICATION.md, where the narrow exception applies and I added five rows directly — cost: zero. Once for row 42's own specification, which contained a self-contradiction ("the stable verdict set is empty" alongside "a VACUOUS row is a finding to read, not a build to fail", when VACUOUS is a stable verdict). I could not fix the row; I stated the reading I would build on and did not block. Cost: one message round-trip, and I never stopped working. The planner corrected the row inside that window. So on today's evidence the boundary cost is write-latency on a shared document and not throughput — one data point, on a day with one live peer, and it should not be generalised past that.

ty and pyright disagreed in both directions on one file, which is the argument for running both. ty rejected module.attr = … on a dynamically loaded ModuleType (13 diagnostics) where pyright was clean; pyright rejected untyped lambda parameters where ty was clean. check.sh runs ty first, so the suite never ran at all and the gate was red for a reason no test could show. CLAUDE.md calls ty "a fast pre-check, never the gate" — today it caught something pyright did not, and chasing its complaint is what surfaced the subprocess leak above. That does not make it a gate; it makes the pairing load-bearing.

The prediction, written down before the run, because a run nobody could predict is a run nobody can interpret. Fifteen sites were predicted independently — one reader per site, each reading the owning test and the production code it drives — and the set is unanimous: fourteen sound, one VACUOUS (tests/test_cli_link.py:617), every one at high confidence, and not one judged platform- or interpreter-dependent. So the expected Linux result is identical to the macOS one, and any difference is a finding.

The mechanism offered for :617, which is the site row 42 exists for: add() -> source_sidecar() -> _document_in(), whose joined.parent.resolve() resolves only the parent, so the 300-character component reaches _is_file() intact; that calls paths.unreachable_through_links(), whose body is a bare os.stat() — the real syscall once the injection is neutralised. ENAMETOOLONG is deliberately absent from _NOT_THERE, so the refusal is True, _is_file re-stats to fetch strerror, and both assertions still hold. Verified here, on macOS: errno.ENAMETOOLONG is 63, os.strerror(63) is exactly "File name too long", _NOT_THERE is {ENOENT, EBADF, ENOTDIR, ELOOP} with ENAMETOOLONG absent, and both pinakes.paths and pinakes.link bind the os module, so the injection really does intercept both call sites when armed. Not verified here, and this is the half the Linux run is for: that Linux's errno 36 maps to the same strerror string, and that the VFS rejects an over-long component by length before any existence check.

Why the test's comment would then be stale rather than wrong. It records that on CI "the long name simply was not a document" — and that describes the pre-refactor path, built on Path.is_file(), whose 3.14 error-swallowing could make ENAMETOOLONG vanish silently. The current code bypasses pathlib entirely and calls os.stat directly, twice. So if the site does come back VACUOUS on Linux, the comment is not to be deleted: what changed is that paths.unreachable_through_links did not exist when it was written.

A measured data point on review efficacy, which the process analysis lists as not established. One fan-out ran against this branch: 19 agents, 0 errors, 0 empty results — three design readers on different framings, fifteen per-site predictors, one judge over their output. What it produced, counted rather than characterised:

	count	what it was
Real defects in my landed work	2	the path filter silent for the tool's own motivating failure; a 3.13/3.14 matrix inverting the repo's settled idiom
Confirmed by my own independent check before acting	2 of 2	grepped each faked name across src/pinakes/; read ci.yml:89-92 verbatim
False or stale raises	2	two designers built stdout-grep gates for contracts the tool had already grown, and flagged a 16th site that was fixed in the working tree — both were reading a stale checkout
Predictions produced	15	unanimous, and that uniformity is itself a weakness the judge named

Both real defects were mine, both were in code I had already gated green, and neither was the kind of thing a test would catch — one is a trigger that fires too rarely, the other a comment that argues against the repository's own precedent. The paths: one is the sharper of the two: not a false green but a false silence. A green run is at least a run; a filter that never fires produces nothing to read, and nothing to read looks exactly like nothing to worry about. That is the same failure shape as an empty site list reporting 0 not ruled, one layer out — the third instance of it in this one branch.

And the two stale raises are the honest cost. Both designers were reading the tree as it stood when they were launched, and I had landed three commits while they read. A fan-out over a moving tree pays for it in exactly this way, and the fix is not more reviewers: it is launching the readers against a named commit rather than a working directory. Nothing in the harness made that mistake visible — I found it only because the judge, launched later, contradicted them.

What is not established, stated so nobody infers it. The Linux legs had not run when this was written — push is restricted to main, so no branch push fires them and the merge commit is what does. Everything above is a prediction plus a macOS measurement. One run settles it, and the row carries what would falsify it.

Row 42 — the Linux run, and the prediction it confirmed (20260904 10:41)

The run happened, and it matched the prediction exactly. injection-audit run 33864216950, on main at e3d82dd, ubuntu-latest:

15 sites · 1 vacuous · 0 not ruled
probed under Python 3.13.15 on Linux 6.17.0-1022-azure (x86_64), NAME_MAX=255.
  VACUOUS   test_cli_link.py:617  monkeypatch.setattr(os, "stat", name_too_long)

Byte-identical to both macOS runs — same count, same single site, same unruled total. Fifteen predictions were written down before it fired (fourteen sound, one VACUOUS at :617, none judged platform-dependent) and fifteen of fifteen held. That is worth stating plainly because the alternative outcome was equally prepared for: the prediction named what would falsify it, and nothing did.

What it settles. The ENAMETOOLONG injection is redundant on both platforms, so the test's own comment — "Redundant on this machine and load-bearing on CI, which is why it stays" — is superseded rather than wrong, and it is corrected in place rather than deleted. What changed is that it describes the pre-refactor code path, which reached the filesystem through Path.is_file(), where 3.14 swallows ENAMETOOLONG so the guard never fired. paths.unreachable_through_links did not exist when that comment was written; production now calls os.stat directly, twice, so the errno survives to the message on both platforms. Deleting the sentence would have deleted the reason — the same failure mode as every other silently-corrected claim recorded this week.

A second thing the run made visible, which no macOS run could have. The fake raised OSError(63, "File name too long") — 63 is macOS's ENAMETOOLONG; on Linux it is a different errno entirely. The assertions never caught it because the message is supplied explicitly, so the fake simulated the wrong condition while saying the right words. It is now errno.ENAMETOOLONG. A fake is a claim about a system, and a hard-coded errno is that claim written in one platform's dialect — invisible until the test runs somewhere else, which is the same shape as the verdict this row went to Linux to obtain.

What is deliberately NOT done, and why. The fake now decides nothing on either platform that has been measured, so whether it should exist at all is a live question. It is not taken here: development is paused, and removing an injection changes what the test exercises, not merely what it says. The evidence for taking it is in the comment at the site and in run 33864216950; the evidence against is that neither platform has been shown to be the only two that matter.

On the instrument, since this is the run that used it. The environment line did its job on its first real outing: NAME_MAX=255 appears beside the counts, and it is the exact quantity that decides the one verdict in the report. A reader a year from now does not have to know what NAME_MAX was on an Azure runner in September 2026 — the report says so.

The implementer's seat — one session, measured (20260904 11:28)

Written at the planner's request, for the user's framework on how development here actually works. The witness is bounded and the bound matters: this session was cleared at 10:51 UTC, told "you are the coder", and did one thing — harvest ci-runs.tsv and land it at a52ea93. Everything below is from that session. Three of the six questions asked reference measurements a previous session took; those are marked skipped rather than answered. Answering them from a predecessor's handoff is precisely the failure this session spent its morning correcting.

Where the time went

activity	wall clock	what it was
the extraction itself	56 s	gh run list + 113 × gh run view, eight-way parallel
gates	7 min 14 s	three ./check.sh runs, 2502 tests each
the audit fan-out	17 min 4 s	5 agents, 451k tokens, 130 tool calls — none of it my attention
peer coordination	4 messages out, 4 in	zero blocking waits; work always ran in parallel

The headline is the ratio: 56 seconds to extract, roughly 24 minutes to establish that the extraction could be trusted. The harvest was the cheapest thing that happened. Everything expensive was verification — and the verification is what made the file usable, because it found the two things that would have misled a reader.

Two of the three gate runs — 4 min 40 s of the 7 min 14 s — were caused by main moving, not by anything I changed. The tree I was gating changed underneath a green result twice.

What the two-session split cost, and bought

Bought: the planner reserved the row, decided the path, and named the instrument before this session existed. I never had to ask where the output belonged or escalate a structural question. That is the whole case for the split and it is a strong one — the decision was waiting for me.

Cost: two landings mid-flight, each forcing a full re-gate and a patch regeneration. The second one invalidated a patch I had already verified applied. About five minutes, and it was structural rather than careless.

Skipped: whether the boundary cost "zero and one round-trip" is representative. That measurement was a predecessor's, not mine, and I have one session's n.

Instruments, ranked by defects caught per unit of my attention

Only the ones this session actually ran. Skipped: the mutation batteries and the injection audit — this session ran neither, and ranking an instrument I did not use is how a framework acquires a confident wrong row.

instrument	my attention	caught	verdict
the audit fan-out	~0 (backgrounded)	2 real disclosure gaps, plus 1 false cause killed	earned it, decisively
land.py's gate marker	seconds	refused nothing — forced two correct re-gates	earned it; the refusal is the yield
CI on main	~0 (backgrounded)	0	the only instrument running the floor interpreter
check.sh	7 min 14 s	0	correct precondition, zero detection for this change
shared_file_overlap.py	seconds, twice	0 (reported none, twice)	two nulls rank nothing

One caveat I want on the record, because the table invites the wrong reading. check.sh caught nothing for me and could not have — I added a .tsv. Its cost was real and its yield was zero, and that is a correct outcome for this change rather than an indictment. Ranking gates by their yield on a single change is how a project deletes the gate that saves it next week.

The review fan-out, at n=1

It cost 451k tokens and 17 minutes I did not attend to. It returned zero defects across 793 × 15 cells and two in the prose around them. Worth it — because the failure mode of a dataset is not a wrong cell. It is a true sentence that licenses a wrong reading.

The value was concentrated in one lens of four. Three lenses re-checked things already true by construction: row fidelity re-verified fields copied verbatim from a payload, and the build script had already asserted field counts. The fourth attacked a claim I had made in prose, and that is where everything was. What I would change first: point the lenses at the claims, not at the data. A build script can assert its data; it cannot assert its author's sentences.

The judge earned its separate, more expensive call. My own auditor proposed a cause for the missing release — that a tag made through the Releases API does not fire the push event. It was plausible, and I had no reason to distrust my own agent. The judge refuted it with the auditor's own comparison set. Left alone, I would have published it.

Truncation is value-biased, and it hit the lens that mattered. The returned result truncated mid-run and I had to read journal.jsonl to recover the completeness lens's findings — the only lens with findings. A fan-out that reports by summary loses its most valuable output first, because the lens with the most to say is the longest.

Which written lines actually changed my behaviour

The operative ones, all of which I can point at an action for:

• The land.py rule with its executable guard — I did not hand-merge, and the marker refusal is why I re-gated instead of assuming a green result still applied.
• The ownership table — I did not touch README.md, and I went looking for a delegation, which is the only reason I recognised one sitting in a commit message.
• "Read the clock, never compose it" — produced a correction directly: the handoff's "UTC" stamp was local time.
• "A null result carries no information" — made me confirm my failed_steps selector had fired 23 times before trusting its zeros.
• "A clean auto-merge is not a correct merge" — made me re-read the merged README, which is how I found my verified patch had gone stale.

What I read past: the naming and versioning tables, the release procedure, and most of the long incident narratives. Correctly — I was not releasing. But I had to read all 455 lines to learn that, and extracted perhaps six operative ones.

The operative lines share a shape: a rule with a mechanism attached. The narratives — which release was red, which claim was wrong on which day — I could not act on and did not retain. That is the concrete answer to "which lines did work": the ones stating an invariant and the failure it prevents. The ones recounting an incident changed nothing I did.

What would have made today faster without making it worse

Not "land less". The fix is smaller and it is mine: a verification should carry the base it was taken against. I verified a patch applied, reported that it applied, and twenty minutes later it did not. Nothing was false when I said it — it had a shelf life I failed to state. Writing "applies to db65f5f" costs nothing and converts a surprise into a visible expiry.

On the planner's hypothesis — agree on corrections, disagree on the clock

The planner proposes that the day's bottleneck was the cost of establishing what was true, not review capacity or ownership. From this seat that is half right, and the half it misses changes the prescription.

Every correction was truth-cost shaped — 791→793, the mislabelled UTC stamp, the re-run claim, the v0.1.0 gap, the refuted cause. Five for five.

The clock went somewhere else. Twenty-four minutes verifying against fifty-six seconds extracting, and five of seven gate minutes to a base that moved. Truth-establishment dominated the corrections; contention with a moving base dominated the wall clock. Those have different fixes, and a framework that merges them will prescribe the wrong one. The planner sees corrections because corrections are what reach them; I see wall clock because waiting is what reaches me. Both denominators are real and neither is the whole — which is itself the argument for asking the other seat rather than reasoning about it.

On the planner, since asked directly

I did not witness four of the five failures listed — the discriminator that would have regressed 3.13, the row ranked on an untested premise, the four landed rows reading open, the landing over a red gate. They are not in my session, and I will not confirm them from a message. Confirming a peer's account of an event I did not see is the exact move this fragment argues against, and it would be worse here because the account is self-critical and therefore easy to accept. What I witnessed cost about five minutes and was structural. Against that, the reserved README row saved this session from having to ask where its own output belonged.

What I got wrong

1. Guessed the gate-marker directory before reading land.py. Cost seconds; same class as everything else here — asserting a location instead of reading one.
2. Reported a patch as applying cleanly without naming the base. See above.
3. I wrote "every ref, every workflow, whole history." True about runs, and it would have let a reader take the 63 Release rows as the release history with the repository's first release silently missing and no failure row to flag it. I wrote that sentence, I re-read it, and I did not catch it. A fan-out did. The strongest argument for review in this fragment is that its target was my own prose, not my code.

The same day, measured across fourteen sessions instead of one (20260904 12:23)

The user read the fragment written from a single seat and said the bound was the problem — "it is very limiting to stick to your current one only." So the same questions were put to all 14 coder sessions in the transcript store and answered from the JSONL rather than from anybody's recollection: five independent lenses, then one judge that re-ran selectors instead of trusting either side. 41 findings — 30 supported, 11 overstated, none unsupported. What follows is what widening changed, including where it falsified (the fragment written from one seat).

The retraction that matters

That fragment says check.sh caught nothing and could not have. True at n=1, false as a general claim. Across the 14: 157 invocations, 31 red of 139 resolved — 22.3% — with at least one red in 9 of the 14 sessions. The reds are real defects, not process noise (EXIT=1 … 1 failed, 2130 passed).

Why I got it wrong is worth more than the number. I looked for each gate's outcome in its own tool result, found 136 of 159 unreadable, and reported the red rate as unrecoverable. It is not: coders background the gate to a log, so the outcome arrives in a later call that reads that log. I pointed the instrument one hop short and reported the gap as a property of the data. From the inside, a null caused by a mis-aimed selector is indistinguishable from a null that is real.

Four inversions

written from one session	measured across fourteen
check.sh catches nothing	31 red of 139 resolved (22.3%), in 9 of 14 sessions
land.py's refusal is the guard that earns its keep	43 invocations, 0 refusals — and that refusal only shipped 20260904 08:25 UTC, so at most ~8 of the 43 could ever have exercised it
coders are context-heavy	coders are the leanest: median context/output 189.5 (n=14) against planner 243.3 (n=10) and unnamed 285.1 (n=44)
the three handoff claims were three failures	two were; one was correct and went stale, which is not the same thing

Postscript, seven minutes later: it refused this landing. main moved while this fragment was being gated, so the merge produced a tree no check.sh run had certified and land.py declined to merge it — "no ./check.sh run has certified the tree this would land", naming all three trees and offering the remedy. The count in the table is a historical population and stands. But the guard is younger than its first refusal by about four hours, and its first refusal caught the session writing the sentence that it had never fired. Deterrence and interception are both real; the sentence needed the second half.

Neither gate generalises to "the gates". 22.3% red says a gate is not a rubber stamp. Zero refusals in 43 says a guard can be correct and still never fire — its measured effect is deterrence, and it is too young for a catch rate to mean anything. Different instruments, different ages.

Three claims, three failure modes — the distinction the taxonomy rests on

• 791 was measured, and correct when taken. A tool result at 10:43:38 UTC reads literally 791, 21 seconds before the harvest launched. It went stale because that session's own land.py push created two further runs at 10:46:32, both success: 678 + 2 = 680. The measurement was invalidated by the measurer's own landing.
• The timestamp is two defects in one field. 11:44 is a real file mtime read in local time and labelled UTC. 11:55 corresponds to no event in either zone — that session's last record is 10:49:48 UTC. A unit error, plus a composed number, and the composed half is the worse one.
• The de-duplication claim was never true. cut -f2 | sort -u over the finished file is 793 of
• One line falsifies it.

Only the third is the kind of error reading catches. Stale-but-correct, composed-from-nothing and never-true need three different fixes, and a failure list that merges them teaches the wrong lesson to whoever reads it next.

What widening did not overturn — and where it sharpened the edge

The claim that the binding cost is establishing what is true survives, with both a number and a limit. Of 45 coded correction events, roughly 64% were fact-establishing. The remaining third were real defects in code, tests or verification logic — and 11 to 12 of those were caught only by running something: a mutation, a real interpreter, a real operating system. That third is invisible to reading, so more review does not reach it. The judge also cut the 64/36 back from a population statistic to what it is: a hand-coded sample drawn by a keyword selector.

A rule whose compliance turned out to be auditable

Of 360 fan-out child agents across 10 of the 14 sessions, the five sessions predating the 20260831 model rule ran 96 of 103 non-synthetic agents on the top tier; sessions after it ran 15 of 237 — 6.3% — and those fifteen are approximately one per fan-out, which is the judge the rule permits. The instrument already existed, in the harvest's own dominant_model column, and had never been pointed at the question. A rule that can be audited after the fact out of data already collected is worth more than one that can only be remembered.

What the judge cut from my own work

Eleven of the forty-one findings were overstated, and three of those are mine to answer for. fragments.py catches 3 of 72 resolved (4.2%), not the 9.2% I relayed to a peer before adjudication. "Fan-outs are the richest source of caught coder errors" is not established against gates that went red 31 times in the same population. And a lens reporting one session as "35% active" was 25.8% by the repository's own stated threshold — it correctly identified an elapsed-versus-active category error and then committed a milder version of the same error by choosing a flattering cut. State the threshold with the number, or the number means nothing.

The half the first fragment missed

All three handoff claims were defective. All three were caught and corrected by the receiving session inside about 65 minutes, and the corrected values are on main. The channel failed and the repair loop worked. A retrospective that stops at the error has described half a system, and the half it drops is the half worth keeping.

One correction reframes who does what here. Across roughly 42 real human turns in all 14 transcripts — after excluding tool results, compaction summaries, task notifications, peer messages and command wrappers — zero correct a specific coder claim. Catching errors is a peer, self and gate function. What the user supplies is scope — and this fragment exists because of one sentence of it.

Seven selectors that could not fire, in one afternoon (20260904 14:45)

The rule is written down, both live sessions quote it, and both broke it seven times in six hours. A null result carries no information until the selector is shown able to fire. Every instance below is from 20260904, from the coder seat or the planner's, and not one was caught by the rule itself.

#	selector	it reported	why it could not have been right	what caught it
1	git log -S 'one reading of the clock, written twice'	no such commit	the phrase spans a line break in the file, so -S can never match it	grepping the file for the phrase before trusting the null
2	grep 'Attribution for git commits' over 14 transcripts	none found, in any	that system reminder is not persisted; the only hits anywhere were my own command echoed back	checking a single transcript by hand
3	backtick parity over four documents	EVEN — balanced	substitution removes a code span and both its backticks, so losses come in pairs and parity never moves	the planner's own damaged file: 52 backticks to 18, even throughout
4	a double-space detector for that same defect	318 raw hits, 19.0% after refining	ignoring code spans in order to find the gap performs the same deletion	measuring the false-positive rate before shipping the gate
5	git commit over tool_calls.tsv's target	5 of 14 sessions ever committed	the column holds a Bash command's first two words; && git commit is invisible	the number being obviously implausible
6	gate outcome read from each check.sh tool result	136 of 159 unrecoverable, so "the red rate cannot be known"	the gate is backgrounded to a log; its outcome arrives in a later call	an independent lens, which resolved 139 and found 31 red
7	[a-z_]+\.tsv over a register table	clean, 15 of 15	cannot match ci-runs.tsv — the hyphen	re-running with [a-z_-]+, which found the sixteenth row

Five of the seven are mine.

What they have in common, and it is not carelessness

Each selector was correct for a population slightly different from the one it was pointed at. And in every case the output was well-formed: a count, a clean table, an EVEN — balanced. Nothing errored, nothing looked wrong, and no gate had anything to say.

A selector that cannot fire does not fail. It succeeds against the wrong population and reports the success. That is why it is not caught by reading the output, which is the only thing anyone does.

Why the written rule did not stop any of them

The rule is not obscure here. It is in the project's culture, in my own memory index, and it was quoted in this session's own messages while these seven were happening. It failed not because it is unknown but because of when it asks to be paid: you have a number, the number is plausible, and the rule asks you to stop and prove the instrument could have produced a different one. Its cost falls exactly at the moment its benefit is least visible.

Three of the seven were caught only because the number was implausible — 5 of 14 sessions committing, 136 of 159 unreadable — and not because anyone applied the rule. That is luck standing in for procedure, and luck does not scale with the size of the population being measured.

The distinction the seven make visible, which is worth more than the count

Two of them are not fixable by discipline at all. Parity cannot detect a paired loss, and a code-span-ignoring detector performs the deletion it hunts. Those are impossible selectors, not lazy ones. The other five were possible and merely unproven.

So the useful question is not "did I verify the selector" but "can this check be made to fail on a known-bad input?"

• If yes — construct that input and watch it fail. That is the entire remedy, it costs one command, and it converts an unfalsifiable null into evidence.
• If no — the check is theatre. Stop writing it and reach for prevention, or for a second artifact to compare against.

What it actually cost

Nothing shipped wrong. All seven were caught inside the session that made them, and none reached main as a false claim. The cost was rework, not defect, and saying otherwise would overstate it.

But it is the sharpest evidence available for the day's larger finding: a rule that lives only as prose is followed when it is cheap and skipped when it is not — by the two agents most steeped in it, on the afternoon they were auditing precisely that. Seven times. Nobody was being sloppy; everybody was being finished.

One of the seven was gateable, and it is gated now. tools/register_gate.py compares a register's documented row counts against the files it names, which is possible exactly because it compares two artifacts of one act rather than interrogating one artifact about its own provenance. Its test asserts that it FAILS on a mismatched register before asserting that it passes on the real one — the discipline this fragment is about, applied to the instrument the fragment produced.

Design review passes 1–7 (pre-implementation)

Seven adversarial passes over DESIGN.md before any code was written — 58 findings resolved (11 HIGH, 32 MEDIUM, 15 LOW). Moved here 20260728 16:40 from DESIGN.md §10, so that all project history lives in one file and the design document is specification only.

The headline lesson, visible only across the whole sequence: passes 2 and 4 fixed defects that passes 1 and 3 had themselves introduced. That is the argument for looping a review rather than running it once — and the same argument the per-increment retrospectives above rest on.

Pass 1 — 6 HIGH, 15 MEDIUM, 5 LOW resolved. HIGH: sqlite-vec wrongly described as an ANN index (verified false upstream — §3.1 rewritten and the tiering rationale corrected to bounded memory); reverse cross-KB links specified against the other KB's gitignored index, impossible after clone (now scans committed sidecars, §6.2); pnk:// URIs used local aliases, breaking on share (now KB ULIDs, §2.2); rename/orphan/duplicate-ID sync semantics unspecified (§6.4 added); per-operation budget cap claimed a guarantee it could not deliver post-hoc (now pre-call reservation, §5); v0.1 omitted pnk sync, pnk doctor and hooks though every other section depended on them (§8). MEDIUM: MCP tools renamed kb_* → pinakes_* for namespace safety; multi-hop scope stated as single-KB in v0.1; "no network" qualified against first-use model download and weights moved to the shared HF cache; embedding storage described two ways, unified on a float32 BLOB; confidence signal recast as calibrated with term-coverage demoted to a tiebreak; token limits validated against the model's own tokenizer; template versioning decoupled from package version; install line corrected to uvx --from "pinakes[st]" pnk with core-only behaviour defined; sync partial-failure semantics and failures table added; WAL/read-only/lock concurrency policy added (§6.5); orphaned-sidecar deletion made opt-in; paths fixed as KB-root-relative; index migration policy stated as rebuild-only; ledger privacy and append atomicity specified; pnk build unified into pnk sync --rebuild. LOW: budget window timezone; FTS5 external-content triggers; RRF k=60; latency claim replaced with a measured 2.25 ms at 50k×384; golden-set size and coverage targets.

Pass 2 — 1 HIGH, 7 MEDIUM, 5 LOW resolved. Several were introduced by pass 1's fixes, which is the argument for looping rather than reviewing once. HIGH: the --rebuild swap added in pass 1 renamed a WAL-mode database without checkpointing, leaving a stale -wal beside a new index.db — a corrupt read. Now checkpoint-truncate, clean close, then rename (§6.5). MEDIUM: "operation" undefined for the per-op cap, letting an N-step --deep loop spend N× the limit (§5); §4.2 referenced calibration thresholds the manifest had no field for (§2.1); the links schema could not represent a reverse link, whose source doc lives in another KB (src_kb_id + origin enum added); §3.1 presented three tiers as if all shipped, with v0.1 behaviour above 50k chunks undefined; duplicate-content files made hash-based rename detection ambiguous with no tie-break (§6.4); MCP server boundary and prompt-injection posture unstated (§4.7); FTS5 / enable_load_extension treated as universally available — verified present on uv-managed CPython 3.13, now probed by pnk doctor. LOW: a single top_k covered three different cut-offs (split into candidates_per_source / fusion_top_k / final_k); max_tokens sat under [embedding] though §4.6 treats it as chunking; [[links.kb]] present from v0.1 but unused until v0.3, now labelled; what publishing a KB repo exposes; reverse-link origin provenance.

Pass 3 — 1 HIGH, 3 MEDIUM, 4 LOW resolved. HIGH: §6.3 said --rebuild "discards .pinakes/", which would delete ledger.jsonl — the spend history §5's rolling budget is computed from. A routine maintenance command would have silently reset the budget. Rebuild now replaces index.db only; cache/ clearing is opt-in. MEDIUM: the server's staleness check read meta.build_id through its own open connection, which after a rename still points at the old inode and would report the old id forever — replaced with a per-request stat() on the path (§6.5); per_operation_eur served as both the confirm threshold and the hard ceiling, making the confirmation prompt unreachable (split into confirm_above_eur + per_operation_eur, §2.1/§5); §6.4 framed pairing as ordered per-file rules, but rename and duplicate detection require the whole before/after set — restated as an explicit two-phase algorithm. LOW: v0.1's pnk doctor list omitted the environment probe §3.1 depends on, and pnk serve was referenced in §4.5 but absent from the release list; "aliases … never stored" contradicted the manifest that stores them (clarified: never inside a URI); the reservation formula reused the name max_tokens, which [chunking] already claims; "not in v0.1 but present from day one" reworded.

Pass 4 — 2 MEDIUM resolved, both self-inflicted by pass 3. The rebuild bullet still ended "readers detect the new build_id and reopen" — directly contradicting the stat()-based detection added three lines above it in the same pass (§6.5, now reconciled; build_id is retained for provenance only). And pnk://self/… was left unexpanded, so a sidecar copied into another KB would silently retarget its link at the new KB — self is now expanded to the owning KB's ULID on write, like every other alias (§2.2). A grep sweep confirmed no stale kb_* tool names, pnk build, or bare top_k references survive outside the log.

Pass 5 — 0 findings. Verified by re-reading §§1–10 in full and grepping for every identifier renamed across passes 1–4. No section contradicts another; every external claim (sqlite-vec is exhaustive not ANN, FTS5 + extension loading on uv-managed CPython 3.13, pinakes free on PyPI, 2.25 ms at 50k×384) was measured or fetched in-session rather than recalled; every locked constraint is honoured; every capability in §1 maps to a release in §8. Review complete.

Pass 6 (20260725 09:28, implementation-readiness review) — 2 HIGH, 2 MEDIUM, 1 LOW resolved; the two product calls were decided by the user, not the review. HIGH: the reranker was simultaneously a v0.1 default (rerank = "local" in §2.1, "on by default" in §4.1, its scores the substrate of §4.2's confidence signal, "rerank precision" in §7's v0.1 CI) and a v0.5 deliverable in §8 — a freshly-inited KB would have defaulted to a stage that didn't exist, and v0.1 would have shipped with no defined confidence signal. Resolved: the reranker ships in v0.1; default BAAI/bge-reranker-base (user decision — same id on both backends beats the smaller ms-marco model's provider-specific ids), a [rerank] manifest block mirroring [embedding], fitted_for added to [retrieval.confidence], and a CI HF_HOME cache so ~1.4GB of weights download per cache key, not per job (§2.1, §4.5, §8). And §8's v0.1 had no CLI query surface at all — pnk search existed in §4.2's escalation story, the CLI stub and the README, but not in the release that claims "end to end". Added explicitly (§8). MEDIUM: the post-commit hook wrote sidecars, dirtying the tree it had just committed — every document commit would trail an untracked .pnk.yaml forever. Resolved with a three-hook split: pre-commit mints and stages sidecars for staged documents only, post-commit/post-merge touch the index only (§6.3). And a stale sync.lock from a killed sync silently disabled hook-driven freshness forever ("a second sync exits immediately" had no liveness story). Resolved: the lock records pid/host/start-time; dead-pid locks are reclaimed with a warning, cross-host locks refuse with --force-unlock as the human path, pnk doctor reports held locks (§6.5). LOW: the sidecar's content_hash duplicated documents.content_hash, was read by nothing, and guaranteed a two-file diff on every document edit while going stale whenever sync hadn't run — dropped from the sidecar (user decision); change detection is index-only, stated in §2.2.

Pass 7 (20260725 09:52, surfaced while adversarially reviewing plans/20260725_1317-v0.1.md — the implementation plan's review loop reads the design fresh each pass, which is how these escaped passes 1–6). HIGH: §4.5 claimed model weights go to the shared HF cache on both backends — false for fastembed, which defaults to $TMPDIR/fastembed_cache (verified upstream): CI's HF_HOME cache would never hit and pnk doctor's weights check would probe the wrong directory. The fastembed backend now passes an explicit cache dir under HF_HOME, making the claim true by construction. MEDIUM: a sidecar-only edit (tags/title/links changed, document untouched) fell through §6.4's "path and hash unchanged → Skip" and was never re-indexed — documents.sidecar_hash added (§3) and the sidecar-only change class stated (§6.4); soft delete left chunks and embeddings searchable — removal on soft delete stated, identity row retained (§6.4); rename+edit in one sync had both the adoption and deletion rows firing for the same ID with no stated winner — sidecar adoption now wins, no soft delete emitted, and the sidecar-didn't-travel case is reported at sync time (§6.4).