fix(verification): a section the runner cannot read SAYS so, and trekplan is pinned to the format it reads
Measured 2026-09-18: nine ordinary shapes of a plan's `## Verification` section parsed to zero criteria - an untagged fence, a ```text fence, a markdown table, `## Verification (acceptance)`, `## Verification:`, `### Verification`, an unclosed fence earlier in the document. Every one came out as `0 of 0`, NOT OK, exit 1, and Phase 7 then forbade `result: completed` without anyone being told that the FORMAT, not the code, was the problem. "The section is empty" and "I cannot read this format" are different facts. Three changes, one hole: - The runner reports `NO_CRITERIA` with a source line (`plan.md:NN`) when the section is there and nothing in it parsed, and names the two forms it does read. Same for a brief's `## Success Criteria`, so the evidence block the conformance reviewer gets says which of the two it is looking at rather than showing an empty table. - Phase 7 says it out loud instead of failing silently: report the source line and the two forms, and say that the plan is what failed there, not the run. - `/trekplan` now pins what it produces to what the runner reads: the heading is exactly `## Verification`, the criteria are a bullet whose first backticked span is the command or a shell-tagged fence, and the command must be one the allowlist runs. A doc-consistency test holds the writer and the reader together, so a runner that learns a new form must update the source. Honest about the round trip: the two round-trip tests were GREEN on arrival - the template already writes the bullet form the runner reads. What was missing was not the format but the PIN: `/trekplan` mandated neither the heading string nor the format, so a plan could satisfy the command's own instructions and still parse to nothing. The tests now hold that. Red first: 5 of the 7 new tests failed before the change (3 NO_CRITERIA, 2 doc-consistency); the 2 round-trip tests are guards, and said so above. Suite 1161 (1159/0/2). Gate unchanged: defects 0 of 7, intact, exit 1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
83d82f121d
commit
f90e1cf02d
5 changed files with 195 additions and 9 deletions
|
|
@ -1917,3 +1917,40 @@ test('D-04: trekreview runs the success-criteria commands and hands the reviewer
|
|||
'the reviewer-launch phase must hand the block over — a block nobody passes is not evidence',
|
||||
);
|
||||
});
|
||||
|
||||
// MAJOR from the 2026-09-18 PM checkpoint: the plan's `## Verification` section
|
||||
// is the one artifact BOTH /trekplan (writer) and lib/verification/criteria-runner.mjs
|
||||
// (reader) touch, and nothing pinned them to the same format. Measured: nine
|
||||
// ordinary shapes of that section parsed to zero criteria, which fells the
|
||||
// single-session run. The writer must pin the heading string and the two forms
|
||||
// the reader reads. Fix the SOURCE: if the runner learns a new form, say so here.
|
||||
test('trekplan pins the ## Verification heading and a format the criteria runner reads', () => {
|
||||
const t = read('commands/trekplan.md');
|
||||
const section = (t.split('8. **Verification**')[1] || '').split('\n9. ')[0];
|
||||
assert.ok(section.length > 0, 'trekplan.md must still carry the Verification instruction');
|
||||
assert.match(
|
||||
section, /exactly `## Verification`/,
|
||||
'the heading string is load-bearing: `## Verification (acceptance)` and `### Verification` parse to nothing',
|
||||
);
|
||||
assert.match(
|
||||
section, /criteria runner|criteria-runner/,
|
||||
'the instruction must name the reader it is writing for',
|
||||
);
|
||||
assert.match(
|
||||
section, /- \[ \] `/,
|
||||
'the bullet form the runner reads must be shown, not described',
|
||||
);
|
||||
assert.match(section, /```bash/, 'the shell-tagged fence form must be shown too');
|
||||
});
|
||||
|
||||
test('trekexecute Phase 7 says NO_CRITERIA out loud instead of failing silently', () => {
|
||||
const phase7 = (read('commands/trekexecute.md').split('\n## Phase 7 — ')[1] || '').split('\n## ')[0];
|
||||
assert.match(
|
||||
phase7, /NO_CRITERIA/,
|
||||
'a section the runner could not read must be reported as such, not as a bare exit 1',
|
||||
);
|
||||
assert.match(
|
||||
phase7, /source line|`\{plan_path\}:/,
|
||||
'the report must carry the line the unreadable section starts on',
|
||||
);
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue