open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

fix(validate): a check that can never run is caught when the brief or plan is written

Browse source 
voyage-lekerepo end-to-end trial (2026-10-06), report points 2, 3, 4, 9:
11 of 13 brief criteria were NOT RUN at review with no warning in /trekbrief;
a plan `git grep` criterion passed validator, gate and three critic rounds,
then felled the run in Phase 7; the End-to-end field and Phase 7's text
search never met; a prose Verify left the test command undefined.

- criteria-runner: runnableCheck() screens a declared check without running
  it (same allowlist; an npm run script is not looked up at write time);
  /bin/bash tests/<script> allowed; the plan report carries end_to_end,
  verified when a passed criterion runs a step's End-to-end command (the
  older text form still counts).
- plan-validator 1.8: PLAN_VERIFICATION_NOT_RUNNABLE,
  PLAN_STEP_VERIFY_NOT_RUNNABLE, PLAN_END_TO_END_NOT_RUNNABLE,
  PLAN_END_TO_END_NOT_VERIFIED (the gate inherits them).
- brief-validator: BRIEF_SC_NOT_RUNNABLE warning; /trekbrief offers a
  tests/ script; brief-reviewer no longer proposes unrunnable commands.
- trekexecute: Phase 7 copies end_to_end from the runner; Verify's test
  command is its first backticked span.

Measured on the trial's own brief and plan: 11 warnings (the 11 NOT RUN),
and the plan stops on V5 git grep and the unlisted End-to-end command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-08 06:42:33 +02:00
commit c56f852ab6
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
14 changed files with 422 additions and 38 deletions
Download patch file Download diff file Expand all files Collapse all files

2
templates/plan-template.md
View file

@ -116,7 +116,7 @@ break, quoted verbatim). A field that is present is never empty.
- File: `path/to/test.ts` *(existing | new)*
- Verifies: {what the test checks}
- Pattern: `path/to/existing-test.ts`
- **End-to-end:** `{the command that runs the product the way a user does — the app, the CLI, the simulator}` → expected: {what the user sees}
- **End-to-end:** `{a command the criteria runner allows that runs the product the way a user does — the app, the CLI, the simulator; usually bash tests/e2e.sh}` → expected: {what the user sees} *(the same command also stands under ## Verification)*
- **Verify:** `{exact command}` → expected: `{output}`
- **On failure:** {revert | retry | skip | escalate} — {specific instructions}
- **Checkpoint:** `git commit -m "{conventional commit message}"`