feat(measure): the yardstick prints Voyage's three numbers with denominators from the source, and it is RED

One command, `node scripts/yardstick.mjs`, prints use, trust and
interrupts. Each row gives countable YES/NO, the value, the denominator and
the source it was read from. A fourth row asks whether the end-state gate can
be fooled by a mutant that keeps the module's signature. Exit is non-zero
while any number is not countable, the mutant is not felled, or the verdict
over the first five deliveries is not judged. Nothing counts a delivery yet,
so today it is RED for the right reasons:

  use         NO   no order in the queue carries work-class
  trust       YES  current ALLOW streak 0 of 37 (BLOCK 14, WARN 12, ALLOW 11,
                   last 2026-09-05 BLOCK, ordered by ts, read as JSON)
  interrupts  NO   0 user_input records of ~141k lines in 5 files, 0 emitter
                   call sites outside lib/stats/ and tests/
  mutants     0 of 1 signature-preserving mutants felled: on M9 (every export
              returns {status:'PASSED'}, every named test only checks typeof)
              the gate closes both D-03 and D-04

It only measures. The gate, the emitter and the order writer are untouched.
The mutant row is meant to stay red until the gate learns to fell M9.

Choices, and why:
- Use counts over the order queue (pending, archive/ and claimed/), because
  an order is the unit of work. The denominator is orders dated on or after a
  cutoff (default 2026-09-21) that carry `work-class: new`. They count as
  having gone through Voyage when a Voyage run in the stats has the order's
  `slug:`. Both fields are the contract the order writer must meet. No order
  carries them today, so the number is NOT COUNTABLE, not 0 %.
- Mailboxes whose names start with a dot are mailboxes too. A shell `*`
  glob skips them, which is why a hand count of `coord/*/orders` comes out
  lower. The row names them so a re-measure can reconcile.
- M9 must be LIVE before its row can count. Its typeof tests have to pass on
  the M9 tree, checked by a runner independent of the gate, so a broken
  fixture can never read as "felled".

Also: tests/fixtures/red-first/02243c6-always-allow-shim.mjs, the shim that
makes the red-first claim of 02243c6 reproducible (6 of 55). The recipe is in
its header.

Red first: the new test file failed to load before scripts/yardstick.mjs
existed, and a dot-mailbox test was red before the census learned about
dot-mailboxes. Suite 1161 -> 1183 (1181/0/2), also green on a clean tree
exported from the index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-22 21:50:06 +02:00
commit 1779411b49
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
3 changed files with 907 additions and 0 deletions

View file

@ -0,0 +1,26 @@
// tests/fixtures/red-first/02243c6-always-allow-shim.mjs
// The shim that makes the red-first claim of 02243c6 reproducible.
//
// 02243c6 ("the criteria runner screens with an ALLOWLIST") says six of its new
// tests were red before the fix. On the parent tree the test file does not even
// LOAD - it imports `allowedCommand`, which the parent does not export - so the
// red is a load error on the whole file, not six named failures. The six only
// appear once the missing export exists and allows everything. This file is
// that export, kept here so a re-measure does not have to guess it.
//
// Reproduce "6 of 55" (POSIX shell, from the repo root, needs git history):
//
// T="$(mktemp -d)"
// git archive 02243c6 | tar -x -C "$T"
// git show 02243c6^:lib/verification/criteria-runner.mjs > "$T/lib/verification/criteria-runner.mjs"
// cat tests/fixtures/red-first/02243c6-always-allow-shim.mjs >> "$T/lib/verification/criteria-runner.mjs"
// (cd "$T" && node --test --test-reporter=tap tests/lib/criteria-runner.test.mjs) | grep -E '^# (tests|fail)'
//
// Expected: `# tests 55` and `# fail 6` - four `allowedCommand` tests (all but
// "every known test runner is allowed", which an always-allow shim passes),
// `runCriteria: a command outside the allowlist is NOT RUN`, and the end-to-end
// canary test (28 evasions, 0 run). Measured 2026-09-22.
// It is a fixture, not a test: nothing in the suite runs it, because the suite
// must also pass on a `git archive` tree, where there is no history to rewind.
export function allowedCommand() { return { allowed: true, reason: '' }; }