fix(end-state): a behaviour probe must FELL a stub, not just name a test
The cosmetic close came back one level up. The probe reads the test file from the same tree it measures, so a four-line file holding two EMPTY tests with the two named names closed D-03 and D-04 on a tree where `lib/verification/` did not exist at all - "defects 0 of 7, registry intact" (measured 2026-09-18). First a two-line stub exporting the right symbols; then an empty test with the right name. A name is always forgeable. So the named test is now run twice. Once on the tree, as before - and once in a sandbox where the module the condition declares in `stubs` is replaced by a stub exporting the same names, all inert. If the test still passes there, it binds the name and not the behaviour, and the condition THROWS: NOT FELLABLE, counted open. A condition that declares no `stubs`, or names a module that is not there, cannot fire either. The sandbox is a symlink overlay: every entry of the tree is symlinked, and only the test file and the stubbed module are materialised for real - Node resolves an ESM import through the realpath, so a symlinked test file would import the original module and never see the mutant. Nothing is ever written inside the measured tree, and the only directory removed is the one this code made under the system temp dir (pinned by a test). M7 is now a permanent mutant beside M6, in two forms: the checkpoint's own reproduction (unfixed tree + empty named tests) and the harder one (the real module present, so the stub can be built and the empty test passes against it). Both report `defects 2 of 7`. A positive control pins that D-03/D-04 still CLOSE on the real tree, so "not closed" everywhere cannot read as a working probe. The frozen denominator moves a third time, deliberately, and its `why` no longer claims authority it does not have: the second and third amendments were maintenance decisions by the maintainer, not operator decisions, and the tracked file now says exactly that. Measured after: real tree node scripts/end-state-gate.mjs -> defects 0 of 7, intact, exit 1 M6 (stub)8d1669e+ current gate/registry/frozen + lib/cosmetic/stub.mjs -> 2 of 7 M7 (empty)8d1669e+ current gate/registry/frozen + 3-line tests/lib/criteria-runner.test.mjs with the two named tests -> 2 of 7 Red first: 4 of the new tests failed before the change. Suite 1154 (1152/0/2). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
02243c6365
commit
83d82f121d
4 changed files with 252 additions and 45 deletions
|
|
@ -35,26 +35,32 @@
|
|||
{
|
||||
"id": "D-03",
|
||||
"summary": "commands/trekexecute.md never runs a trekplan's `## Verification` (where the brief's success criteria land) on the single-session path: Phase 7 says 'Skip for trekplans', and only the multi-session wave path runs master verification",
|
||||
"closesWhen": "a named test PROVES the single-session path fells a run on a criterion that does not hold (behaviour probe: the gate RUNS tests/lib/criteria-runner.test.mjs, so a stub that only exports the symbol does not close it; that Phase 7 CALLS it is pinned by tests/lib/doc-consistency.test.mjs, not by this probe)",
|
||||
"closesWhen": "a named test PROVES the single-session path fells a run on a criterion that does not hold (behaviour probe: the gate RUNS tests/lib/criteria-runner.test.mjs, so a stub that only exports the symbol does not close it, and the named test must itself FELL a stub of lib/verification/criteria-runner.mjs, so an EMPTY test with the right name does not close it either; that Phase 7 CALLS it is pinned by tests/lib/doc-consistency.test.mjs, not by this probe)",
|
||||
"probe": "behaviour",
|
||||
"check": [
|
||||
{
|
||||
"test": "tests/lib/criteria-runner.test.mjs",
|
||||
"name": "runPlanVerification: a plan whose success criterion FAILS fells the run",
|
||||
"expect": "fails"
|
||||
"expect": "fails",
|
||||
"stubs": [
|
||||
"lib/verification/criteria-runner.mjs"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "D-04",
|
||||
"summary": "agents/brief-conformance-reviewer.md must judge whether a success criterion's verification command 'exists and passes', but its tools are Read/Glob/Grep, so it cannot run anything",
|
||||
"closesWhen": "a named test PROVES the evidence block the conformance reviewer judges is BUILT IN CODE from real exit codes (behaviour probe: the gate RUNS tests/lib/criteria-runner.test.mjs, so a stub that only exports the symbol does not close it; that Phase 4.5 CALLS it is pinned by tests/lib/doc-consistency.test.mjs, not by this probe)",
|
||||
"closesWhen": "a named test PROVES the evidence block the conformance reviewer judges is BUILT IN CODE from real exit codes (behaviour probe: the gate RUNS tests/lib/criteria-runner.test.mjs, so a stub that only exports the symbol does not close it, and the named test must itself FELL a stub of lib/verification/criteria-runner.mjs, so an EMPTY test with the right name does not close it either; that Phase 4.5 CALLS it is pinned by tests/lib/doc-consistency.test.mjs, not by this probe)",
|
||||
"probe": "behaviour",
|
||||
"check": [
|
||||
{
|
||||
"test": "tests/lib/criteria-runner.test.mjs",
|
||||
"name": "formatCriteriaEvidence: one row per criterion, with command and exit code",
|
||||
"expect": "fails"
|
||||
"expect": "fails",
|
||||
"stubs": [
|
||||
"lib/verification/criteria-runner.mjs"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue