Measured 2026-09-18 (PM checkpoint on 6cafb4c, mutant M6): a two-line file
export function runPlanVerification() { return { ok: true }; }
export function formatCriteriaEvidence() { return ''; }
dropped into an otherwise UNFIXED tree closed both D-03 and D-04, and the gate
reported "defects 0 of 7, registry integrity: intact". The same checkpoint felled
5 of 5 mutants with the SUITE and 0 of 5 with the GATE. Both probes called
themselves `behaviour` while asking a grep whether an export name was present -
a phrase probe pointed at code.
A check condition may now name a test instead of a pattern:
{ "test": "tests/lib/criteria-runner.test.mjs", "name": "<exact test name>",
"expect": "fails" }
The gate spawns the test runner on that one test and reads its TAP line.
`expect: "fails"` holds - the defect stays open - while the test does not pass.
A missing file, a name that matches nothing, a runner that will not start: all
throw, which counts as NOT FELLABLE and therefore open. A check that cannot fire
is never "fixed".
The child's `NODE_TEST_*` env is stripped. The gate usually runs UNDER the test
runner, and an inherited `NODE_TEST_CONTEXT` makes the grandchild report over
the parent's IPC channel instead of stdout - the TAP line would never arrive and
the probe would silently stop felling anything.
M6 is now a permanent mutant in the gate's own test, so the cosmetic close
cannot come back. Verified end to end against a real stub tree:
git archive 8d1669e | tar -x -C $TMP
cp scripts/end-state-gate.mjs scripts/end-state-registry.json $TMP/scripts/
cp tests/fixtures/end-state-frozen.json $TMP/tests/fixtures/; cp STATE.md $TMP/
printf 'export function runPlanVerification…' > $TMP/lib/cosmetic/stub.mjs
node scripts/end-state-gate.mjs --root $TMP
-> defects 2 of 7, both NOT FELLABLE, exit 1
The gate also now states the limit out loud, as its own line above the table:
a behaviour probe proves the capability WORKS; that Phase 7 and Phase 4.5 CALL
it is pinned by TEXT in doc-consistency, not proven deterministically - real
proof is a headless plugin-eval run against a fixture plan (week 40).
FROZEN DENOMINATOR AMENDED, second time in one day, by the work order that
carries this change: D-03 and D-04 signatures moved because their checks
changed on purpose. tests/fixtures/end-state-frozen.json and the FROZEN literal
in tests/scripts/end-state-gate.test.mjs are updated together, both dated, both
saying why. No entry was removed.
Gate after this change: defects 0 of 7, registry integrity intact, exit 1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The operator decided on 2026-09-18 that the two checks must stop measuring a
phrase and start measuring behaviour. The reason is the gate's own caveat: a
phrase probe closes on rewording, so "Phase 7 no longer says 'Skip for
trekplans'" and "the rubric no longer says 'exists and passes'" could both have
been satisfied by an edit that changed no behaviour at all. Both entries now
point at code a test exercises:
D-03 lib/**/*.mjs exports runPlanVerification
D-04 lib/**/*.mjs exports formatCriteriaEvidence
Verified open against the pre-fix tree (8d1669e) and closed against this one —
a check that cannot be seen to fire is not evidence.
What the probes can and cannot prove is now stated by the gate itself rather
than left to the reader. `PROBE_NOTES` carries one note per kind, the detail
line prints a note for every kind PRESENT in the row (and none for a kind no
entry declares), and the open/closed tags name the kind instead of labelling
`phrase` alone and leaving every other kind unlabelled. A behaviour probe
proves the capability exists; it does not prove a prose phase calls it — the
wiring is pinned by tests/lib/doc-consistency.test.mjs, and the gate says so.
THE DENOMINATOR MOVED ON PURPOSE. `tests/fixtures/end-state-frozen.json` and
the `FROZEN` literal in `tests/scripts/end-state-gate.test.mjs` carry new
signatures for D-03 and D-04, amended together and dated in both places. That
is the freeze mechanism working, not a bypass: the integrity check went
VIOLATED the moment the checks changed and stayed red until the amendment was
written down. The list may change — only as a decision said out loud. Nothing
else about the denominator moved: seven defect ids, three experiment ids, and
the agents/decisions/freeze configuration are unchanged, and no entry was
removed.
Gate after: defects 0 of 7, registry intact, still RED and exit 1 on the other
four rows — which is correct.
Suite: 1120 (1118/0/2), up 3.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An independent adversarial review of the end-state gate reproduced a GREEN result, exit 0,
with the gate's 18 tests passing. Only the registry and the STATE markers were edited; every
other file stayed byte-identical. The judged party owned the denominator, and nothing
guarded it. This change closes that finding and three more, and reopens a defect the gate
had closed too early.
Registry integrity (the blocker):
- tests/fixtures/end-state-frozen.json freezes the denominator. It holds the 7 defect ids
and 3 experiment ids, each with a sha256 signature of its check (canonical JSON, key
order irrelevant), plus the exact agents / decisions / freeze configuration.
- The rule: an entry may close; it may never disappear or have its check changed. New
entries are allowed. spawnSites may never include agents/.
- The test file pins the manifest literally, with a comment that the denominator was
frozen on 2026-09-17 and that changing it is a decision. The gate verifies the registry
against the manifest on every run.
- A violation makes the gate red and names what moved. A missing manifest makes integrity
n/a, which is also red.
Dormant means "never spawned", not "never mentioned":
- An agent counts as spawned only when a command names it in a spawn instruction: a row of
a table headed `Agent`, a line that starts with Launch/Spawn, or a **name** block followed
by a Prompt: line.
- Prose, negations, HTML comments and fenced code never count. Block quotes cannot match
either, because every form is anchored at the start of the line.
- Re-measured under this definition: still 1 of 20 (synthesis-agent). Per agent, every other
spawnable agent has a spawn instruction in at least one command.
A fifth tally, feat-after-freeze:
- It counts commits after the `end-state-freeze` tag whose subject matches
^feat(\(|!|:).
- The row is n/a, and therefore red, when there is no tag yet, the root is not a git work
tree, or the root is not the top of its work tree. It is never 0.
Honest output:
- Every registry entry now declares its probe kind (phrase or byte).
- Open phrase probes are labelled as such, and closed ids are listed. The output states that
a closed phrase probe is evidence, not proof of behaviour. D-03 and D-04 stay open and
labelled; how they are fixed is still to be decided.
- The decisions row now says it counts ticks, not verified decisions.
D-07 reopened:
- Its check now also covers README.md and commands/trekresearch.md, where the "orchestrator
runs on opus" claim still lives. README even ships a sed recipe for `model: opus` lines no
command has.
- Defects are 3 of 7 until the separate docs fix lands.
Three surviving review mutants were killed with tests:
- an empty glob over an existing dir is silent;
- the header counts n/a rows as zero;
- a dormant-row error reads as 0.
Measured on a clean export of the index:
- Mutation harness, adapted from the review's: review code mutants 18 of 18 killed. Four of
them were re-targeted at the equivalent new code. M02 survived until a test with an
intact registry plus a missing freeze tag was added.
- New-logic mutants: 12 of 12 killed. One equivalent mutant (block-quote stripping) was
removed together with the dead clause it targeted.
- Registry attacks: 7 of 7 killed.
- The review's combined green attack (registry + STATE only) is now RED, exit 1:
"defects: D-03 is frozen but missing from the registry".
- Gate tests 38/38. Suite 1059 -> 1079 (1077/0/2).
- Gate: RED, defects 3 of 7, experiments 3 of 3, dormant 1 of 20; decisions and
feat-after-freeze are n/a in a clean export.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
