voyage/tests/fixtures
Kjell Tore Guttormsen 1779411b49
feat(measure): the yardstick prints Voyage's three numbers with denominators from the source, and it is RED
One command, `node scripts/yardstick.mjs`, prints use, trust and
interrupts. Each row gives countable YES/NO, the value, the denominator and
the source it was read from. A fourth row asks whether the end-state gate can
be fooled by a mutant that keeps the module's signature. Exit is non-zero
while any number is not countable, the mutant is not felled, or the verdict
over the first five deliveries is not judged. Nothing counts a delivery yet,
so today it is RED for the right reasons:

  use         NO   no order in the queue carries work-class
  trust       YES  current ALLOW streak 0 of 37 (BLOCK 14, WARN 12, ALLOW 11,
                   last 2026-09-05 BLOCK, ordered by ts, read as JSON)
  interrupts  NO   0 user_input records of ~141k lines in 5 files, 0 emitter
                   call sites outside lib/stats/ and tests/
  mutants     0 of 1 signature-preserving mutants felled: on M9 (every export
              returns {status:'PASSED'}, every named test only checks typeof)
              the gate closes both D-03 and D-04

It only measures. The gate, the emitter and the order writer are untouched.
The mutant row is meant to stay red until the gate learns to fell M9.

Choices, and why:
- Use counts over the order queue (pending, archive/ and claimed/), because
  an order is the unit of work. The denominator is orders dated on or after a
  cutoff (default 2026-09-21) that carry `work-class: new`. They count as
  having gone through Voyage when a Voyage run in the stats has the order's
  `slug:`. Both fields are the contract the order writer must meet. No order
  carries them today, so the number is NOT COUNTABLE, not 0 %.
- Mailboxes whose names start with a dot are mailboxes too. A shell `*`
  glob skips them, which is why a hand count of `coord/*/orders` comes out
  lower. The row names them so a re-measure can reconcile.
- M9 must be LIVE before its row can count. Its typeof tests have to pass on
  the M9 tree, checked by a runner independent of the gate, so a broken
  fixture can never read as "felled".

Also: tests/fixtures/red-first/02243c6-always-allow-shim.mjs, the shim that
makes the red-first claim of 02243c6 reproducible (6 of 55). The recipe is in
its header.

Red first: the new test file failed to load before scripts/yardstick.mjs
existed, and a dot-mailbox test was red before the census learned about
dot-mailboxes. Suite 1161 -> 1183 (1181/0/2), also green on a clean tree
exported from the index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-22 21:50:06 +02:00
..
bakeoff feat(voyage): S10 — NW2 part A (Workflow port + fidelity harness + smoke) 2026-06-18 14:40:19 +02:00
bakeoff-rich feat(eval): SKAL-1·4b offline gold-scored output eval 2026-06-30 09:00:33 +02:00
red-first feat(measure): the yardstick prints Voyage's three numbers with denominators from the source, and it is RED 2026-09-22 21:50:06 +02:00
session-state test(ultraplan-local): add session-state fixtures 2026-05-01 20:21:50 +02:00
synthesis feat(voyage): S12 — NW3 synthesis-agent built + measured → declined per measurement [skip-docs] 2026-06-18 17:58:39 +02:00
token-usage feat(stats): add pure token-usage parser + cache-aware cost derivation 2026-06-26 14:32:10 +02:00
trekreview feat(voyage)!: bulk content rewrite ultra -> voyage/trek prose [skip-docs] 2026-05-05 15:08:20 +02:00
brief-effort-fable.md test(validators): cover fable accept + unknown-model reject in both gate layers 2026-07-02 16:56:26 +02:00
brief-effort-high.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-effort-low.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-effort-standard.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-success-criteria.md fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
brief-v21-no-signals.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-with-phase-signals.md feat(voyage): bump trekbrief-template to brief_version 2.1 + add phase_signals fixtures 2026-05-13 21:09:57 +02:00
brief-without-phase-signals.md feat(voyage): bump trekbrief-template to brief_version 2.1 + add phase_signals fixtures 2026-05-13 21:09:57 +02:00
criteria-exit-0.sh fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
criteria-exit-1.sh fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
end-state-frozen.json fix(end-state): a behaviour probe must FELL a stub, not just name a test 2026-09-18 02:56:22 +02:00
expected.prom feat(trekresearch): emit and export the five STORM measurement fields 2026-08-09 15:01:15 +02:00
jsonl-schemas.md fix(storm-measure): check BOTH halves of the activation SC, not just the count delta 2026-08-12 23:01:59 +02:00
plan-fase-narrative.md chore(release): ultraplan-local v3.1.0 2026-05-01 06:31:42 +02:00
plan-profile-drift.md feat(voyage): emit MANIFEST_PROFILE_DRIFT warning in plan-validator strict mode — brief assumption 7 2026-05-09 10:02:53 +02:00
plan-verification-fails.md fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
plan-verification-passes.md fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
plan-with-profile.md feat(voyage): add lib/profiles/resolver.mjs — locked interface SC #5-#9 2026-05-09 09:29:01 +02:00
plan-without-profile.md feat(voyage): add lib/profiles/resolver.mjs — locked interface SC #5-#9 2026-05-09 09:29:01 +02:00
profile-invalid-enum.yaml feat(voyage): add lib/validators/profile-validator.mjs — SC #1, #2, #3 2026-05-09 09:26:23 +02:00
profile-invalid-model.yaml feat(voyage): add lib/validators/profile-validator.mjs — SC #1, #2, #3 2026-05-09 09:26:23 +02:00
research-deep-research-adapted.md test(trekresearch): pin deep-research adapter output contract 2026-06-30 13:38:09 +02:00
stats-sample.jsonl feat(trekresearch): emit and export the five STORM measurement fields 2026-08-09 15:01:15 +02:00
stats-with-profile.jsonl feat(voyage): add 5 additive profile fields to JSONL stats — SC #11 2026-05-09 09:40:21 +02:00