test(readme): the selftest numbers rot loudly, against the suites' own summaries
Order 20260905T053602Z-6743615726-from-.claude (.claude, 2026-09-05), asking for the check this repo recommended when it re-measured the README a run earlier. The badge and the five `## Development` comments rotted twice in a row - 529 carried from 0.25.0, then a badge saying 868 beside comments summing to 792, two different wrong sums of the same fact on the same screen - because nothing compared them to anything. It lives in tests/selftest.test.mjs, not in a bash suite, and the choice was measured rather than assumed. The order's parenthetical pointed at whichever suite already pins README/catalog invariants; no such suite exists - `grep -ln README scripts/*selftest*.sh` returns board-selftest.sh alone, on two incidental hits (a prose comment and a research/README.md fixture). This wrapper is the only place where all five numbers exist at once in a run that already happens. runSuite() captures each suite's own summary line, so the truth source is the line the suite prints. A check inside one suite could see its own total but would have to RE-RUN the other four - 212s sequentially, measured 2026-09-05 under /bin/bash 3.2 (coord 16, board 169, route 12, orders 5, guard 10) - and grepping `check` calls out of the scripts is both the second copy of the counting the order warned against and a wrong one, since those calls sit inside loops. Three properties are deliberate. The badge is compared against the MEASURED sum, never against the five README comments: a badge agreeing with five stale comments is the 868-beside-792 shape one layer down. A suite that stops printing a recognisable summary FAILS the check rather than being skipped - an absent measurement must not read as a matching one. And the check adds no bash check anywhere, so the five counts and the 893 badge are unchanged by its arrival, exactly as the order expects; a counted self-check would have had to compare against PASS+FAIL+1 and would break for whoever adds a check after it. Ground truth on this HEAD, run before anything was written: coord 257, board 393, route 73, orders 116, guard 54 = 893, 0 failed in all five - the README was already correct, so the red step is the mutation. Mutation-verified in both directions: 73 -> 74 on the route comment gives "README says route-selftest has 74 checks; it reported 73"; 893 -> 894 on the badge gives "README's badge says 894 selftest checks; the five suites reported 893"; restored, npm test is 12/12 green. Bounded gap, stated rather than closed: CLAUDE.md's own copies of the five counts are a second surface carrying the same numbers and are NOT checked. Measured and left alone - widening the check to it was not ordered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
9a154950eb
commit
a1ef1fb555
3 changed files with 127 additions and 8 deletions
32
CLAUDE.md
32
CLAUDE.md
|
|
@ -1464,6 +1464,36 @@ obligations in another repo.
|
|||
and re-measured after: 11 of 11 now match. Only the Install block was
|
||||
converted; converting the other 39 indented blocks was not ordered and
|
||||
would bury a two-line fix in a whole-file diff.
|
||||
- **README's selftest numbers are pinned by `tests/selftest.test.mjs`, and the
|
||||
check lives there rather than in one of the five bash suites for a measured
|
||||
reason.** The badge and the five `## Development` comments rotted twice in a
|
||||
row - 529 carried from 0.25.0, then a badge saying 868 beside comments summing
|
||||
to 792, two different wrong sums of the same fact on the same screen - because
|
||||
nothing compared them to anything. The order behind the fix
|
||||
(`20260905T053602Z-6743615726`) suggested hosting it in whichever suite
|
||||
already pins README/catalog invariants; measured before choosing, **no such
|
||||
suite exists** (`grep -ln README scripts/*selftest*.sh` returns
|
||||
`board-selftest.sh` alone, on two incidental hits - a prose comment and a
|
||||
`research/README.md` fixture). This wrapper is the only place where all five
|
||||
numbers exist at once in a run that already happens: `runSuite()` captures
|
||||
each suite's own summary line, so the truth source is the line the suite
|
||||
prints, never a re-count. A check inside one suite could see its own total but
|
||||
would have to RE-RUN the other four (**212s sequentially**, measured
|
||||
2026-09-05 under `/bin/bash` 3.2) to see theirs, and grepping `check` calls out
|
||||
of the scripts is both the second copy of the counting and a wrong one, since
|
||||
those calls sit inside loops. The badge is compared against the MEASURED sum,
|
||||
not against the five README comments - a badge agreeing with five stale
|
||||
comments is the 868-beside-792 shape one layer down - and a suite that stops
|
||||
printing a summary FAILS the check rather than being skipped, because an
|
||||
absent measurement must not read as a matching one. It adds no bash check, so
|
||||
the five counts and the 893 badge are unchanged by its arrival.
|
||||
Mutation-verified in both directions: `73 -> 74` on the route comment gives
|
||||
"README says route-selftest has 74 checks; it reported 73", `893 -> 894` on
|
||||
the badge gives "README's badge says 894 selftest checks; the five suites
|
||||
reported 893", and the restored file is 12/12 green. **Bounded gap, stated
|
||||
rather than closed:** `CLAUDE.md`'s own copies of the five counts (in this
|
||||
section, above) are NOT checked - a second surface carrying the same numbers,
|
||||
measured and left alone because widening the check to it was not ordered.
|
||||
- Conventional Commits: `type(scope): description`.
|
||||
|
||||
## Commands
|
||||
|
|
@ -1471,7 +1501,7 @@ obligations in another repo.
|
|||
- Test: `bash scripts/coord-selftest.sh`, `bash scripts/board-selftest.sh`,
|
||||
`bash scripts/route-selftest.sh`, `bash scripts/orders-selftest.sh` and
|
||||
`bash scripts/state-line-guard-selftest.sh` (or `npm test`, the Node wrapper
|
||||
around all five plus the hook tests)
|
||||
around all five plus the hook tests and the README-number check)
|
||||
- Order queue smoke test: `CLAUDE_COORD_DIR=$(mktemp -d) bash
|
||||
scripts/coord-order-send.sh --to smoke --from tester --subject s --message m`
|
||||
then `CLAUDE_COORD_DIR=<same> bash scripts/coord-order-inbox.sh --repo smoke`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue