test(readme): the selftest numbers rot loudly, against the suites' own summaries

Order 20260905T053602Z-6743615726-from-.claude (.claude, 2026-09-05), asking
for the check this repo recommended when it re-measured the README a run
earlier. The badge and the five `## Development` comments rotted twice in a
row - 529 carried from 0.25.0, then a badge saying 868 beside comments summing
to 792, two different wrong sums of the same fact on the same screen - because
nothing compared them to anything.

It lives in tests/selftest.test.mjs, not in a bash suite, and the choice was
measured rather than assumed. The order's parenthetical pointed at whichever
suite already pins README/catalog invariants; no such suite exists -
`grep -ln README scripts/*selftest*.sh` returns board-selftest.sh alone, on two
incidental hits (a prose comment and a research/README.md fixture). This
wrapper is the only place where all five numbers exist at once in a run that
already happens. runSuite() captures each suite's own summary line, so the
truth source is the line the suite prints. A check inside one suite could see
its own total but would have to RE-RUN the other four - 212s sequentially,
measured 2026-09-05 under /bin/bash 3.2 (coord 16, board 169, route 12,
orders 5, guard 10) - and grepping `check` calls out of the scripts is both the
second copy of the counting the order warned against and a wrong one, since
those calls sit inside loops.

Three properties are deliberate. The badge is compared against the MEASURED
sum, never against the five README comments: a badge agreeing with five stale
comments is the 868-beside-792 shape one layer down. A suite that stops
printing a recognisable summary FAILS the check rather than being skipped -
an absent measurement must not read as a matching one. And the check adds no
bash check anywhere, so the five counts and the 893 badge are unchanged by its
arrival, exactly as the order expects; a counted self-check would have had to
compare against PASS+FAIL+1 and would break for whoever adds a check after it.

Ground truth on this HEAD, run before anything was written: coord 257,
board 393, route 73, orders 116, guard 54 = 893, 0 failed in all five - the
README was already correct, so the red step is the mutation. Mutation-verified
in both directions: 73 -> 74 on the route comment gives "README says
route-selftest has 74 checks; it reported 73"; 893 -> 894 on the badge gives
"README's badge says 894 selftest checks; the five suites reported 893";
restored, npm test is 12/12 green.

Bounded gap, stated rather than closed: CLAUDE.md's own copies of the five
counts are a second surface carrying the same numbers and are NOT checked.
Measured and left alone - widening the check to it was not ordered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

CLAUDE.md

@@ -1464,6 +1464,36 @@ obligations in another repo.
   and re-measured after: 11 of 11 now match. Only the Install block was
   converted; converting the other 39 indented blocks was not ordered and
   would bury a two-line fix in a whole-file diff.
+- **README's selftest numbers are pinned by `tests/selftest.test.mjs`, and the
+  check lives there rather than in one of the five bash suites for a measured
+  reason.** The badge and the five `## Development` comments rotted twice in a
+  row - 529 carried from 0.25.0, then a badge saying 868 beside comments summing
+  to 792, two different wrong sums of the same fact on the same screen - because
+  nothing compared them to anything. The order behind the fix
+  (`20260905T053602Z-6743615726`) suggested hosting it in whichever suite
+  already pins README/catalog invariants; measured before choosing, **no such
+  suite exists** (`grep -ln README scripts/*selftest*.sh` returns
+  `board-selftest.sh` alone, on two incidental hits - a prose comment and a
+  `research/README.md` fixture). This wrapper is the only place where all five
+  numbers exist at once in a run that already happens: `runSuite()` captures
+  each suite's own summary line, so the truth source is the line the suite
+  prints, never a re-count. A check inside one suite could see its own total but
+  would have to RE-RUN the other four (**212s sequentially**, measured
+  2026-09-05 under `/bin/bash` 3.2) to see theirs, and grepping `check` calls out
+  of the scripts is both the second copy of the counting and a wrong one, since
+  those calls sit inside loops. The badge is compared against the MEASURED sum,
+  not against the five README comments - a badge agreeing with five stale
+  comments is the 868-beside-792 shape one layer down - and a suite that stops
+  printing a summary FAILS the check rather than being skipped, because an
+  absent measurement must not read as a matching one. It adds no bash check, so
+  the five counts and the 893 badge are unchanged by its arrival.
+  Mutation-verified in both directions: `73 -> 74` on the route comment gives
+  "README says route-selftest has 74 checks; it reported 73", `893 -> 894` on
+  the badge gives "README's badge says 894 selftest checks; the five suites
+  reported 893", and the restored file is 12/12 green. **Bounded gap, stated
+  rather than closed:** `CLAUDE.md`'s own copies of the five counts (in this
+  section, above) are NOT checked - a second surface carrying the same numbers,
+  measured and left alone because widening the check to it was not ordered.
 - Conventional Commits: `type(scope): description`.
 
 ## Commands
@@ -1471,7 +1501,7 @@ obligations in another repo.
 - Test: `bash scripts/coord-selftest.sh`, `bash scripts/board-selftest.sh`,
   `bash scripts/route-selftest.sh`, `bash scripts/orders-selftest.sh` and
   `bash scripts/state-line-guard-selftest.sh` (or `npm test`, the Node wrapper
-  around all five plus the hook tests)
+  around all five plus the hook tests and the README-number check)
 - Order queue smoke test: `CLAUDE_COORD_DIR=$(mktemp -d) bash
   scripts/coord-order-send.sh --to smoke --from tester --subject s --message m`
   then `CLAUDE_COORD_DIR=<same> bash scripts/coord-order-inbox.sh --repo smoke`

README.md

@@ -200,7 +200,7 @@ Note that raising the inbox's priority (Rule 7) deliberately does **not** widen
     bash scripts/route-selftest.sh              # 73 checks, incl. the route->board round trip
     bash scripts/orders-selftest.sh             # 116 checks, incl. the 20-way barriered claim race
     bash scripts/state-line-guard-selftest.sh   # 54 checks, incl. the Edit replace_all projection and the ratchet
-    npm test                                    # all five selftests plus the hook tests, via node --test
+    npm test                                    # all five selftests, the hook tests, and the README-number check
 
 TDD is the house rule: every behavior change lands with a failing selftest check first.
 

tests/selftest.test.mjs

@@ -4,7 +4,7 @@
 import { test } from 'node:test';
 import assert from 'node:assert';
 import { execFileSync } from 'node:child_process';
-import { mkdtempSync, mkdirSync, writeFileSync, existsSync } from 'node:fs';
+import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, existsSync } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { basename, dirname, join } from 'node:path';
 import { fileURLToPath } from 'node:url';
@@ -12,8 +12,31 @@ import { fileURLToPath } from 'node:url';
 const root = join(dirname(fileURLToPath(import.meta.url)), '..');
 const hook = join(root, 'hooks', 'scripts', 'session-start.mjs');
 
+// Every bash suite already prints its own total on its last line, and this
+// wrapper already runs all five. Capturing that line here is what makes the
+// README's numbers testable without a SECOND copy of the counting: nothing
+// re-counts `check` calls (loops make that undecidable anyway) and nothing
+// re-runs a suite to read a number the run in progress is already printing.
+// The five suites cost 212s sequentially, measured 2026-09-05 under /bin/bash
+// 3.2 - the marginal cost of the README check is zero because it consumes a
+// run that happens regardless.
+const summaries = new Map();
+
+function runSuite(name) {
+  const script = join(root, 'scripts', `${name}-selftest.sh`);
+  try {
+    summaries.set(name, execFileSync('bash', [script], { encoding: 'utf8' }));
+  } catch (err) {
+    // Record what the suite managed to print before failing, then let the
+    // failure through: a red suite must stay red here, and the README check
+    // below still gets a number to compare rather than a silent absence.
+    if (typeof err.stdout === 'string') summaries.set(name, err.stdout);
+    throw err;
+  }
+}
+
 test('coord bash selftest passes', () => {
-  execFileSync('bash', [join(root, 'scripts', 'coord-selftest.sh')], { encoding: 'utf8' });
+  runSuite('coord');
 });
 
 // board.sh reads this plugin's mailbox for its INN column, so the board ships
@@ -22,7 +45,7 @@ test('coord bash selftest passes', () => {
 // through CLAUDE_PLUGIN_ROOT, so a board.sh that exists only in
 // ~/.claude/scripts/ would be missing on exactly the path production uses.
 test('board bash selftest passes', () => {
-  execFileSync('bash', [join(root, 'scripts', 'board-selftest.sh')], { encoding: 'utf8' });
+  runSuite('board');
 });
 
 // route.sh is the WRITER for the next-cost field board.sh already reads, so its
@@ -31,7 +54,7 @@ test('board bash selftest passes', () => {
 // CLAUDE_PLUGIN_ROOT, and a calculator proven only elsewhere is unproven on the
 // one path production uses.
 test('route bash selftest passes', () => {
-  execFileSync('bash', [join(root, 'scripts', 'route-selftest.sh')], { encoding: 'utf8' });
+  runSuite('route');
 });
 
 // pre-state-line-guard.mjs is a PreToolUse hook, so like session-start.mjs it
@@ -39,7 +62,7 @@ test('route bash selftest passes', () => {
 // CLAUDE_PLUGIN_ROOT, and a guard proven only elsewhere is unproven on the
 // path production actually runs.
 test('state-line-guard bash selftest passes', () => {
-  execFileSync('bash', [join(root, 'scripts', 'state-line-guard-selftest.sh')], { encoding: 'utf8' });
+  runSuite('state-line-guard');
 });
 
 // The order queue is the second channel beside the mailbox, with the opposite
@@ -48,7 +71,7 @@ test('state-line-guard bash selftest passes', () => {
 // CLAUDE_PLUGIN_ROOT, so a queue proven only elsewhere is unproven where it
 // runs.
 test('orders bash selftest passes', () => {
-  execFileSync('bash', [join(root, 'scripts', 'orders-selftest.sh')], { encoding: 'utf8' });
+  runSuite('orders');
 });
 
 // The engine refuses to invent an identity from the cwd, but the hook is the
@@ -188,3 +211,69 @@ test('hook keeps mail and orders in separate blocks, mail first', () => {
   assert.ok(ctx.includes('UNTRUSTED DATA'), 'the mail block lost its authorization framing');
   assert.ok(ctx.includes('OPERATOR-AUTHORIZED'), 'the order block lost its authorization framing');
 });
+
+// --- README's selftest numbers must rot loudly ------------------------------
+//
+// The badge and the five `## Development` comments are the only public claim
+// about how much this engine is pinned by, and they are the number furthest
+// from the meter: they rotted twice in a row (529 from 0.25.0; then a badge
+// saying 868 beside comments summing to 792 - two different wrong sums of the
+// same fact, neither matching the other, on the same screen). Nothing caught
+// either, because nothing compared them to anything.
+//
+// It lives HERE rather than in one of the five bash suites, and the choice is
+// not arbitrary. The order's parenthetical suggested the suite that already
+// pins README/catalog invariants; measured before choosing, no such suite
+// exists - `grep -ln README scripts/*selftest*.sh` returns board-selftest.sh
+// alone, on two incidental hits (a prose comment and a `research/README.md`
+// fixture). Of the places that could host it, this wrapper is the only one
+// where all five numbers exist at once in a run that already happens: a check
+// inside a suite could see its own count but would have to RE-RUN the other
+// four (212s, measured 2026-09-05) to see theirs, and reading counters out of
+// the scripts is the second copy of the counting this check was asked not to
+// be. `check` calls sit inside loops, so a static count is not merely a second
+// copy - it is a wrong one.
+//
+// The truth source is each suite's own summary line, verbatim, and a suite
+// that stops printing one FAILS here rather than being skipped: an absent
+// measurement must not read as a matching one.
+function suiteTotal(name) {
+  const out = summaries.get(name);
+  assert.ok(out !== undefined,
+    `${name}-selftest produced no captured output: its total was never measured, ` +
+    'so the README comparison below would be resting on nothing');
+  // Two summary grammars, both already in the tree: coord prints
+  // `PASS=N FAIL=M`, the other four print `<name>-selftest: N passed, M failed`
+  // and orders adds `, S skipped (of T checks)`. README documents the TOTAL
+  // number of checks, so skipped ones count.
+  let m = out.match(/^\S+-selftest: (\d+) passed, (\d+) failed(?:, (\d+) skipped)?/m);
+  if (m) return Number(m[1]) + Number(m[2]) + Number(m[3] ?? 0);
+  m = out.match(/^PASS=(\d+) FAIL=(\d+)/m);
+  assert.ok(m, `${name}-selftest printed no summary line this parser recognises`);
+  return Number(m[1]) + Number(m[2]);
+}
+
+test('README states the selftest counts the suites actually reported', () => {
+  const readme = readFileSync(join(root, 'README.md'), 'utf8');
+  const suites = ['coord', 'board', 'route', 'orders', 'state-line-guard'];
+
+  let sum = 0;
+  for (const name of suites) {
+    const measured = suiteTotal(name);
+    sum += measured;
+    const line = readme.match(
+      new RegExp(`^\\s*bash scripts/${name}-selftest\\.sh\\s+#\\s+(\\d+) checks`, 'm'));
+    assert.ok(line,
+      `README's ## Development block has no "N checks" comment for ${name}-selftest.sh`);
+    assert.equal(Number(line[1]), measured,
+      `README says ${name}-selftest has ${line[1]} checks; it reported ${measured}`);
+  }
+
+  // The badge is the sum, and it is compared against the MEASURED total rather
+  // than against the five README comments: a badge agreeing with five stale
+  // comments is exactly the 868-beside-792 shape, one layer down.
+  const badge = readme.match(/badge\/selftest_checks-(\d+)-/);
+  assert.ok(badge, 'README has no selftest_checks badge to check');
+  assert.equal(Number(badge[1]), sum,
+    `README's badge says ${badge[1]} selftest checks; the five suites reported ${sum}`);
+});