voyage/tests/validators
Kjell Tore Guttormsen c56f852ab6
fix(validate): a check that can never run is caught when the brief or plan is written
voyage-lekerepo end-to-end trial (2026-10-06), report points 2, 3, 4, 9:
11 of 13 brief criteria were NOT RUN at review with no warning in /trekbrief;
a plan `git grep` criterion passed validator, gate and three critic rounds,
then felled the run in Phase 7; the End-to-end field and Phase 7's text
search never met; a prose Verify left the test command undefined.

- criteria-runner: runnableCheck() screens a declared check without running
  it (same allowlist; an npm run script is not looked up at write time);
  /bin/bash tests/<script> allowed; the plan report carries end_to_end,
  verified when a passed criterion runs a step's End-to-end command (the
  older text form still counts).
- plan-validator 1.8: PLAN_VERIFICATION_NOT_RUNNABLE,
  PLAN_STEP_VERIFY_NOT_RUNNABLE, PLAN_END_TO_END_NOT_RUNNABLE,
  PLAN_END_TO_END_NOT_VERIFIED (the gate inherits them).
- brief-validator: BRIEF_SC_NOT_RUNNABLE warning; /trekbrief offers a
  tests/ script; brief-reviewer no longer proposes unrunnable commands.
- trekexecute: Phase 7 copies end_to_end from the runner; Verify's test
  command is its first backticked span.

Measured on the trial's own brief and plan: 11 warnings (the 11 NOT RUN),
and the plan stops on V5 git grep and the unlisted End-to-end command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 06:42:33 +02:00
..
architecture-discovery.test.mjs feat(voyage)!: bulk content rewrite ultra -> voyage/trek prose [skip-docs] 2026-05-05 15:08:20 +02:00
brief-ceremony.test.mjs feat(brief): ceremony by size, one question round, logged and committed approvals 2026-10-06 19:44:42 +02:00
brief-gate-coverage.test.mjs test(eval): two-sided gate coverage for BRIEF_* BLOCKERs 2026-06-26 11:57:10 +02:00
brief-validator.test.mjs test(validators): cover fable accept + unknown-model reject in both gate layers 2026-07-02 16:56:26 +02:00
intent-approval.test.mjs test(intent): M3–M7 — halt pinned where it acts, allowlist guarded, header claims pinned 2026-09-23 09:23:04 +02:00
next-session-prompt-validator.test.mjs feat(voyage)!: rename produced_by field values + validator comments [skip-docs] 2026-05-05 14:42:21 +02:00
plan-gate.test.mjs fix(validate): a check that can never run is caught when the brief or plan is written 2026-10-08 06:42:33 +02:00
plan-validator-profile-drift.test.mjs feat(voyage): emit MANIFEST_PROFILE_DRIFT warning in plan-validator strict mode — brief assumption 7 2026-05-09 10:02:53 +02:00
plan-validator-v18.test.mjs fix(validate): a check that can never run is caught when the brief or plan is written 2026-10-08 06:42:33 +02:00
plan-validator.test.mjs fix(voyage): S26 — parse plan_version prose form (S22 defect #2/#3) 2026-06-19 21:59:31 +02:00
profile-validator.test.mjs feat(profiles): add built-in fable profile (all six phases on fable) 2026-07-02 16:59:26 +02:00
progress-validator.test.mjs fix(validators): the progress validator knows stopped and blocked 2026-10-05 17:32:54 +02:00
query-privacy-gate.test.mjs fix(validators): hard-block the token formats the run-length patterns missed 2026-08-12 22:03:43 +02:00
research-validator.test.mjs feat(voyage)!: rename type discriminators across validators + fixtures [skip-docs] 2026-05-05 14:40:25 +02:00
review-validator.test.mjs fix(review): close the gate's fail-open paths an independent review found 2026-10-06 20:03:16 +02:00
runnable-checks.test.mjs fix(validate): a check that can never run is caught when the brief or plan is written 2026-10-08 06:42:33 +02:00
session-state-validator.test.mjs feat(voyage)!: bulk content rewrite ultra -> voyage/trek prose [skip-docs] 2026-05-05 15:08:20 +02:00