voyage/lib/verification
Kjell Tore Guttormsen 0013d292d6
fix(verification): the criteria runner reads the repo's own plan format
The runner read bullet lines only. The repo's own example plan writes its
whole acceptance run as a fenced bash block, so `## Verification` parsed to
ZERO criteria, the runner exited 1, and Phase 7 forbade `result: completed`
- a correct plan felled every single-session run. Measured 2026-09-18 on
6cafb4c: none of the repo's plan artifacts exited 0.

Fences are now read twice over, for two opposite reasons:

- a `## ` heading INSIDE a fence is quoted text and no longer opens a
  section. examples/02-real-cli/REGENERATED.md is a report that quotes a
  plan outline inside one fence; it used to yield an empty section that
  read as "0 of 0", and now yields the honest NO_VERIFICATION_SECTION.
- a shell-tagged fence inside the section holds the commands. The tag list
  is closed (bash/sh/shell/zsh/console/shell-session): an untagged fence is
  more often expected OUTPUT than input, and inventing a criterion from
  output is the failure this file exists to prevent.

Blank and comment-only lines inside the block declare nothing. A `$`/`>`
console prompt is stripped; a `#` root prompt is NOT, because it cannot be
told from a comment and running a comment is the worse mistake.

Measured after the fix (parse only - one example names a fictional CLI):
examples/01 6 criteria, plan-template 2 (both placeholders, correctly NOT
RUN), the two runner fixtures 2 each, plan-run-C 1, REGENERATED.md 0 with
NO_VERIFICATION_SECTION.

Divergence from the order's premise, stated for the record: it said 2 of 3
example plans write `## Verification` as a fenced bash block. Ground truth
is 1 of 3 - REGENERATED.md has no section of its own at all - and the
example plan's block holds 6 command lines, not 5.

Red first: 4 of the 6 new tests failed before this change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 02:02:14 +02:00
..
criteria-runner.mjs fix(verification): the criteria runner reads the repo's own plan format 2026-09-18 02:02:14 +02:00