fix(execute): run the plan's Verification on the single-session path (D-03)
A trekplan's `## Verification` section is where the brief's success criteria land. Phase 7 opened with "**Skip for trekplans.**", and only the multi-session wave path (Phase 2.6 Step 3) ran master verification. A plan executed in ONE session therefore reported `completed` without ever running the criteria it was measured against — the executor's own belief was the only evidence. The check now exists as code, not as an instruction: - `lib/verification/criteria-runner.mjs` parses the criteria an artifact DECLARES (a plan's `## Verification`, a brief's `## Success Criteria`), runs each command, and returns a verdict built from exit codes. Fail-closed throughout: a placeholder, a prose-only criterion, or an unavailable screen is `unrunnable`/`blocked`, never `passed`. A plan with no `## Verification` section exits 1 — a plan that promises no end-to-end check cannot be reported as verified. - Every command is screened through the plugin's own PreToolUse denylist (`hooks/scripts/pre-bash-executor.mjs`) before it reaches a shell. Chose invoking that hook over its documented stdin protocol rather than copying its rules, because a command spawned from node never passes through the Bash tool and so the hook cannot fire by itself — this keeps exactly one denylist. - Phase 7 is now "Exit / verification check": session specs run the exit condition, trekplans run the criteria runner. Phase 4's entry-condition skip for trekplans stands — a plan carries no entry condition; the exit side is not symmetrical. - A failing criterion FELLS the run: `plan_verification.status != "passed"` forbids `result: completed`. That is clause 2 of the stop-signal contract, now enforced on the single-session path too. Red first: `tests/lib/criteria-runner.test.mjs` (26 tests) against two committed fixture plans, one of which declares a criterion that fails on purpose. The doc pin in `tests/lib/doc-consistency.test.mjs` guards the wiring — a capability no phase calls is the same defect wearing a lib/ file; verified red against the pre-fix Phase 7 (skip present, runner absent, no fell-the-run clause). Suite: 1108 (1106/0/2), up 27 from 1081. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
8d1669ef51
commit
e55ca9fc89
6 changed files with 756 additions and 6 deletions
25
tests/fixtures/plan-verification-fails.md
vendored
Normal file
25
tests/fixtures/plan-verification-fails.md
vendored
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
---
|
||||
task: criteria-runner fixture — one verification criterion fails
|
||||
slug: criteria-runner-fixture-fail
|
||||
---
|
||||
|
||||
# Plan: criteria-runner fixture (one criterion FAILS on purpose)
|
||||
|
||||
Fixture only. Consumed by `tests/lib/criteria-runner.test.mjs`. It is the
|
||||
falsifying case for D-03: a plan whose declared success criterion does not
|
||||
hold must fell the single-session run instead of being noted and passed over.
|
||||
|
||||
## Steps
|
||||
|
||||
### Step 1 — nothing
|
||||
|
||||
Fixtures have no steps worth running.
|
||||
|
||||
## Verification
|
||||
|
||||
- [ ] `true` -> expected: exit 0
|
||||
- [ ] `false` -> expected: exit 0 (FAILS on purpose — exit 1)
|
||||
|
||||
## Estimated Scope
|
||||
|
||||
- **Files to modify:** 0
|
||||
24
tests/fixtures/plan-verification-passes.md
vendored
Normal file
24
tests/fixtures/plan-verification-passes.md
vendored
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
---
|
||||
task: criteria-runner fixture — every verification criterion passes
|
||||
slug: criteria-runner-fixture-pass
|
||||
---
|
||||
|
||||
# Plan: criteria-runner fixture (all criteria pass)
|
||||
|
||||
Fixture only. Consumed by `tests/lib/criteria-runner.test.mjs`; never executed
|
||||
by the pipeline. The commands are deliberately trivial and side-effect free.
|
||||
|
||||
## Steps
|
||||
|
||||
### Step 1 — nothing
|
||||
|
||||
Fixtures have no steps worth running.
|
||||
|
||||
## Verification
|
||||
|
||||
- [ ] `true` -> expected: exit 0
|
||||
- [ ] `printf ok` -> expected: `ok` on stdout, exit 0
|
||||
|
||||
## Estimated Scope
|
||||
|
||||
- **Files to modify:** 0
|
||||
Loading…
Add table
Add a link
Reference in a new issue