New § Stability: why case 4 was unstable (it could halt on the intent gate,
5 of 6 here before the fix, 6 of 12 and 7 of 10 in the PM re-measurement),
the three changes, the case-4 known-positive (3 of 3 passed with the gate's
code off until names-code; then 0 of 3), and the says-fail cause left
unmeasured (the failing trace was deleted; 0 of 13 kept replies used
another spelling).
After the fix (a384ce6): three full runs (one --runs 3, two --runs 1),
every case 5 of 5, each full run 8 of 8; case 4 alone 10 of 10, 15 of 15
in total. The status line no longer lets 8 of 8 read as a stable state.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v2 = eight cases, expectations committed in 3892a83 before the run.
Run 1: 8/8 cases, 36/36 graders, 65 s, $1.97 (list-price estimate).
Known-positive for the intent-gate case, in a scratch copy: --check made
to accept everything → 0.75 (names-code fails). Removing only the gate's
command line from trekplan.md did NOT fell it (1.00): the child rebuilt the
call from the surrounding prose — the case measures the gate's effect; the
exact line is pinned by the unit test that executes it. One green run of
case 5 does not prove the instability gone; that needs more runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/proevesett.md records what the suite covers and what it does not, how to
run it, the verbatim pass/fail definition with its source, and the first run:
6 of 7 cases (graders 25 of 26), 7 claude -p children, 54 s.
The red case, review-requires-project, lost only its text grader: the child
paraphrased a guard that lives in prose. A diagnostic re-run passed, so it is
unstable, not broken, and it stays red - loosening the pattern after the run
would fit the grader to the result.
Known-positive shown in a scratch copy: with the finding-ID check disabled in
review-validator.mjs, review-validate-flags-bad-finding-id drops from 1.0 to 0.6.
Also recorded, measured by probe: ${CLAUDE_PLUGIN_ROOT} is substituted in the
expanded command text but empty in the child's Bash env, and the plugin
directory is readable from the child's sandbox.
Veikart step 4 uses the suite as its yardstick: a rule change that lowers n stops.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
