fix(execute): run the plan's Verification on the single-session path (D-03)

A trekplan's `## Verification` section is where the brief's success criteria
land. Phase 7 opened with "**Skip for trekplans.**", and only the multi-session
wave path (Phase 2.6 Step 3) ran master verification. A plan executed in ONE
session therefore reported `completed` without ever running the criteria it was
measured against — the executor's own belief was the only evidence.

The check now exists as code, not as an instruction:

- `lib/verification/criteria-runner.mjs` parses the criteria an artifact
  DECLARES (a plan's `## Verification`, a brief's `## Success Criteria`), runs
  each command, and returns a verdict built from exit codes. Fail-closed
  throughout: a placeholder, a prose-only criterion, or an unavailable screen
  is `unrunnable`/`blocked`, never `passed`. A plan with no `## Verification`
  section exits 1 — a plan that promises no end-to-end check cannot be reported
  as verified.
- Every command is screened through the plugin's own PreToolUse denylist
  (`hooks/scripts/pre-bash-executor.mjs`) before it reaches a shell. Chose
  invoking that hook over its documented stdin protocol rather than copying its
  rules, because a command spawned from node never passes through the Bash tool
  and so the hook cannot fire by itself — this keeps exactly one denylist.
- Phase 7 is now "Exit / verification check": session specs run the exit
  condition, trekplans run the criteria runner. Phase 4's entry-condition skip
  for trekplans stands — a plan carries no entry condition; the exit side is
  not symmetrical.
- A failing criterion FELLS the run: `plan_verification.status != "passed"`
  forbids `result: completed`. That is clause 2 of the stop-signal contract,
  now enforced on the single-session path too.

Red first: `tests/lib/criteria-runner.test.mjs` (26 tests) against two committed
fixture plans, one of which declares a criterion that fails on purpose. The
doc pin in `tests/lib/doc-consistency.test.mjs` guards the wiring — a capability
no phase calls is the same defect wearing a lib/ file; verified red against the
pre-fix Phase 7 (skip present, runner absent, no fell-the-run clause).

Suite: 1108 (1106/0/2), up 27 from 1081.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-18 01:30:57 +02:00
commit e55ca9fc89
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
6 changed files with 756 additions and 6 deletions

View file

@ -897,10 +897,17 @@ progress file.
"iterations_remaining": 25,
"entry_condition_checked": false,
"exit_condition_checked": false,
"plan_verification": null,
"summary": null
}
```
**`plan_verification` (additive-optional):** written by Phase 7 when the input
is a trekplan — `{status, summary, failed_criteria}`, where `status` is
`"passed" | "failed" | "not-run"`. `null` for session specs (they have an exit
condition instead) and on early stop. Unknown keys are tolerated by
`progress-validator.mjs`, so a legacy progress file without it still validates.
**`iterations_remaining` (v1.7+, additive-optional):** the remaining
recovery/retry budget against the global hard cap (unit = **iterations**, per the
brief — token/cost budgeting is out of scope). Seeded to
@ -1225,9 +1232,24 @@ Update progress: `steps.{N}.status = "passed"`, `steps.{N}.commit = {hash}`,
If mode = `step N`: after completing step N (pass or fail), skip remaining steps
and jump to Phase 8 (final report).
## Phase 7 — Exit condition check (session specs only)
## Phase 7 — Exit / verification check
**Skip for trekplans.** Run only when all steps passed (not on early stop).
**Runs only when all steps passed** (not on early stop). Which check runs
depends on what was executed:
| Input | Check |
|-------|-------|
| session spec | the `## Exit Condition` checklist |
| trekplan | the plan's `## Verification` criteria |
Phase 4's entry-condition skip for trekplans stands — a plan carries no entry
condition. The exit side is not symmetrical: a trekplan's `## Verification`
section is where the brief's success criteria land, and it used to run only on
the multi-session wave path (Phase 2.6 Step 3). A single-session run therefore
reported `completed` without ever executing the criteria it was measured
against. It no longer does.
### Session specs — exit condition checklist
Run each exit condition command from the `## Exit Condition` checklist:
@ -1240,6 +1262,62 @@ Exit condition check:
If all pass: `exit_condition_checked: true` in progress file.
If any fail: record which failed. Include in final report.
### trekplans — run the plan's `## Verification`
Run the criteria runner over the plan file parsed in Phase 1
(`{plan_path}`). It parses the `## Verification` section, screens every command
through the plugin's own PreToolUse denylist before it reaches a shell, and
runs what survives — in the FOREGROUND, never backgrounded. (A command spawned
from node does not pass through the Bash tool, so that hook cannot fire by
itself; the runner invokes it over its documented stdin protocol instead of
carrying a second copy of the denylist.)
```bash
# Resolve the plugin root ONCE. ${CLAUDE_PLUGIN_ROOT} is substituted in this
# command's text but is EMPTY in the Bash tool's process env, and a bare
# `node ${CLAUDE_PLUGIN_ROOT}/lib/…` then runs `node /lib/…`, which exits 1 —
# indistinguishable from a criterion that failed.
VOYAGE_ROOT="${CLAUDE_PLUGIN_ROOT:-}"
case "$VOYAGE_ROOT" in
/*) ;;
*) VOYAGE_ROOT="$(ls -d "$HOME"/.claude/plugins/cache/*/voyage 2>/dev/null | head -1)" ;;
esac
if [ ! -f "$VOYAGE_ROOT/lib/verification/criteria-runner.mjs" ]; then
echo "[voyage] plan verification could not run - plugin root unresolved (exit 2)."
echo " NOT a pass: report it and never emit result: completed."
exit 2
fi
node "$VOYAGE_ROOT/lib/verification/criteria-runner.mjs" --plan "{plan_path}" --json
```
The exit code is the verdict, and it is not advisory:
| Exit | Meaning | Executor |
|------|---------|----------|
| 0 | every criterion passed | `plan_verification.status = "passed"`; continue |
| 1 | a criterion failed, was blocked, or could not run | `plan_verification.status = "failed"`; progress `status: "failed"` |
| 2 | the runner itself could not run | `plan_verification.status = "not-run"`; treated exactly like exit 1 |
A plan with no `## Verification` section exits 1 with
`error.code = NO_VERIFICATION_SECTION`. That is deliberate: a plan that
promises no end-to-end check cannot be reported as verified.
Record in the progress file (additive-optional; unknown keys are tolerated by
`progress-validator.mjs`, so a legacy progress file still validates):
- `plan_verification.status` — `"passed" | "failed" | "not-run"`
- `plan_verification.summary` — `{total, passed, failed, blocked, unrunnable}`
- `plan_verification.failed_criteria` — `[{label, command, exit_code}]`
**A failing criterion FELLS the run.** It is not "recorded and included in the
final report": the executor MUST NOT emit `result: completed` while
`plan_verification.status != "passed"`. This is clause 2 of the stop-signal
contract below (Phase 7.5) — "the plan's Verify command(s) passed (exit 0)" —
now enforced on the single-session path and not only on the multi-session one.
Continue to Phase 7.5 either way: the manifest audit is independent, and a
verification failure must not hide a manifest drift.
## Phase 7.5 — Manifest audit (independent)
**Runs for all modes except dry-run.** This is the last-line-of-defense
@ -1301,7 +1379,11 @@ Record in progress file:
> 0)** / lint clean / an explicit **DONE token emitted AFTER the Phase 7.5
> audit ran** (never before, never in place of it). A transcript that merely
> "feels done" is NOT a stop-signal — victory-declaration bias is the exact
> failure mode this gate exists to defeat.
> failure mode this gate exists to defeat. For a trekplan this clause is
> **Phase 7's** `plan_verification.status == "passed"` — a real exit code
> from `lib/verification/criteria-runner.mjs`, on the single-session path as
> well as the multi-session one. `"failed"` or `"not-run"` OVERRIDES
> `completed` to `failed`, exactly as the audit-override does.
> 3. **Counter liveness (deterministic, defeats "never decremented")** — the gate
> reconciles the budget counter against the ground-truth counters the Phase 7.5
> audit already derives:
@ -1524,6 +1606,12 @@ Phase 2.3 (validate exit) and Phase 5 (dry-run) intentionally do not write
| 2 | {desc} | FAIL | 3 | — | — |
| 3 | {desc} | — | 0 | — | — |
### Plan Verification (Phase 7, trekplans only)
- **Status:** {passed | failed | not-run | n/a}
- **Criteria:** {passed}/{total} passed, {failed} failed, {blocked} blocked, {unrunnable} not run
- **Failed criteria:** {label + command + exit code, one per line; empty when passed}
### Manifest Audit (Phase 7.5)
- **Status:** {pass | drift}
@ -1540,8 +1628,8 @@ Phase 2.3 (validate exit) and Phase 5 (dry-run) intentionally do not write
- Not reached: {N}
- Blocked (sandbox): {N}
{if all passed + exit condition passed}:
All steps completed. Exit condition: PASS.
{if all passed + exit condition passed + plan verification passed}:
All steps completed. Exit condition: PASS. Plan verification: PASS.
{if failed/stopped}:
### Failure Details
@ -1559,7 +1647,8 @@ To resume: /trekexecute --resume {path}
```
**Result vocabulary (v1.7, strict):**
- `completed` — all steps passed AND Phase 7.5 manifest audit passed
- `completed` — all steps passed AND Phase 7.5 manifest audit passed AND
(for trekplans) Phase 7 plan verification passed
- `partial` — steps passed per executor but Phase 7.5 found drift, OR
Phase 7.6 recovery incomplete
- `blocked` — Step 0 sandbox pre-flight exited 77; no real work attempted
@ -1583,6 +1672,7 @@ To resume: /trekexecute --resume {path}
"steps_blocked": 0,
"failed_at_step": null,
"exit_condition": "{pass | fail | skipped | n/a}",
"plan_verification": "{passed | failed | not-run | n/a}",
"manifest_audit": "{pass | drift | n/a}",
"drift_details": [],
"recovery_dispatched": false,