fix(validate): a check that can never run is caught when the brief or plan is written

voyage-lekerepo end-to-end trial (2026-10-06), report points 2, 3, 4, 9:
11 of 13 brief criteria were NOT RUN at review with no warning in /trekbrief;
a plan `git grep` criterion passed validator, gate and three critic rounds,
then felled the run in Phase 7; the End-to-end field and Phase 7's text
search never met; a prose Verify left the test command undefined.

- criteria-runner: runnableCheck() screens a declared check without running
  it (same allowlist; an npm run script is not looked up at write time);
  /bin/bash tests/<script> allowed; the plan report carries end_to_end,
  verified when a passed criterion runs a step's End-to-end command (the
  older text form still counts).
- plan-validator 1.8: PLAN_VERIFICATION_NOT_RUNNABLE,
  PLAN_STEP_VERIFY_NOT_RUNNABLE, PLAN_END_TO_END_NOT_RUNNABLE,
  PLAN_END_TO_END_NOT_VERIFIED (the gate inherits them).
- brief-validator: BRIEF_SC_NOT_RUNNABLE warning; /trekbrief offers a
  tests/ script; brief-reviewer no longer proposes unrunnable commands.
- trekexecute: Phase 7 copies end_to_end from the runner; Verify's test
  command is its first backticked span.

Measured on the trial's own brief and plan: 11 warnings (the 11 NOT RUN),
and the plan stops on V5 git grep and the unlisted End-to-end command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-08 06:42:33 +02:00
commit c56f852ab6
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
14 changed files with 422 additions and 38 deletions

View file

@ -56,6 +56,17 @@ Flag as **incomplete** if:
- Success Criteria are not testable (e.g., "it should work well")
- Scope is unbounded — no non-goals defined
**Runnable at review.** `/trekreview` runs a criterion's command only when it
is one test-runner invocation with no pipe, redirect or `&&`: `npm test`,
`npm run <script>`, `node --test <file>`, `vitest`, `jest`, `pytest`,
`python -m pytest`, `uv run pytest`, `cargo test`, `go test`, `make test`,
`bash tests/<script>.sh` (or `/bin/bash tests/<script>.sh`), or `git status|log|diff|show|ls-files`.
Anything else — the product's own binary (`node bin/x.mjs …`), `jq`, `grep`,
`git grep`, a pipe — is reported NOT RUN. Never PROPOSE such a command as a
criterion. When a criterion needs one, propose wrapping it in a script under
`tests/` and writing the criterion as "`bash tests/<script>.sh` exits 0".
The validator flags existing ones as `BRIEF_SC_NOT_RUNNABLE`.
### 2. Consistency
Check for internal contradictions: