fix(validate): a check that can never run is caught when the brief or plan is written
voyage-lekerepo end-to-end trial (2026-10-06), report points 2, 3, 4, 9: 11 of 13 brief criteria were NOT RUN at review with no warning in /trekbrief; a plan `git grep` criterion passed validator, gate and three critic rounds, then felled the run in Phase 7; the End-to-end field and Phase 7's text search never met; a prose Verify left the test command undefined. - criteria-runner: runnableCheck() screens a declared check without running it (same allowlist; an npm run script is not looked up at write time); /bin/bash tests/<script> allowed; the plan report carries end_to_end, verified when a passed criterion runs a step's End-to-end command (the older text form still counts). - plan-validator 1.8: PLAN_VERIFICATION_NOT_RUNNABLE, PLAN_STEP_VERIFY_NOT_RUNNABLE, PLAN_END_TO_END_NOT_RUNNABLE, PLAN_END_TO_END_NOT_VERIFIED (the gate inherits them). - brief-validator: BRIEF_SC_NOT_RUNNABLE warning; /trekbrief offers a tests/ script; brief-reviewer no longer proposes unrunnable commands. - trekexecute: Phase 7 copies end_to_end from the runner; Verify's test command is its first backticked span. Measured on the trial's own brief and plan: 11 warnings (the 11 NOT RUN), and the plan stops on V5 git grep and the unlisted End-to-end command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
ca6ee8779a
commit
c56f852ab6
14 changed files with 422 additions and 38 deletions
|
|
@ -73,7 +73,7 @@ test('K3: no critic verdict → 1 (the plan has not been through the critic)', (
|
|||
});
|
||||
|
||||
test('K3: an invalid plan → 1 even with a clean verdict', () => {
|
||||
const bad = buildPlan({ steps: [{ testFirst: false }, { endToEnd: 'node x' }] });
|
||||
const bad = buildPlan({ steps: [{ testFirst: false }, { endToEnd: 'bash tests/e2e.sh' }] });
|
||||
const g = gatePlan(rec(bad, dedupOut()).text);
|
||||
assert.equal(g.code, 1);
|
||||
assert.ok(g.reasons.some(x => x.code === 'PLAN_GATE_INVALID_PLAN'));
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue