fix: round 9 of the release review of 6.3.0, for 6.3.1

Round 9 (0f400d3..a10018f, WARN: 0 BLOCKER, 12 MAJOR): seven findings and
three brief notes, by operator choice. Each finding began with a test that
failed on a10018f on its assertion; suite 2230 (2228/0/2).

- A check refused for its comment (a quote, $, backtick or backslash) keeps
  its command and its reason, and every reader refuses it out loud: a nested
  ## Verification bullet stays a criterion (PLAN_VERIFICATION_NOT_RUNNABLE)
  and a brief SC warns (BRIEF_SC_NOT_RUNNABLE). It was folded into a
  sub-point or read as prose, a regression from a10018f. Readers that took a
  command as runnable (the end-to-end lists) skip a refused one.
- The verdict gate reads the last fenced block's info string and body from
  one reading, and exits 2 unless the info string is exactly json: a last
  block opened as ```json {x} held a BLOCKER the gate passed with exit 0
  (also in 6.3.0). A last block never closed is exit 2 too.
- One fence reader (findings-schema fencedBlocks) for the gate and every
  trailing-json reader: a fence closes only on a run at least as long as its
  opener, as in Markdown, so a four-backtick example of a json block is one
  block; CRLF and an indented fence read alike in the gate and the readers.
- Files: a description runs on past a comma and onto the next line until a
  new entry starts (a list item on a new line, or a backticked path), and a
  plain path followed by a colon starts its description. Measured over all
  939 Files: fields in the .md files under the marketplace and the test repo
  (560 of them in the 48 plan.md files outside worktrees): 0 read
  differently, tests-only changed in 0.
- The test lock compares --cmd by its words, so other blanks between the
  same words are the step's command.
- release-check decides "two files" on real paths: one review.md reached
  through a symlinked --repo is one file.
- Brief notes: red proof's stopped control run has its own test (green at
  once, red with the branch deleted); the two tests that assumed jest is not
  installed run with a PATH that finds only git (red on a10018f with a jest
  on PATH, green now); the Files: reader's doc states the measured counts
  (520 fields in 48 plan files at rounds 7-8, not "182 plans"; 560 now).

Not in this round (operator 2026-10-11): a144e9ce, 1dc2e785, dc3a467e,
2fa88853, 26873fa5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-11 12:49:01 +02:00
commit a1e0e50451
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
11 changed files with 416 additions and 104 deletions

View file

@ -288,8 +288,12 @@ export function onFailureAction(text) {
* The paths a step's `Files:` field names, in order, for the tests-only check
* below (the K6 sweep keeps v6.2.0's own reader: plan-scope-sweep.mjs,
* operator 2026-10-11). The convention in the plan template is a backticked
* path, and in every plan measured (520 Files: fields in 182 plans, 11.10)
* a field with backticked paths had no other path-like word beside them:
* path. Measured on the plan.md files outside worktrees under the marketplace
* and the test repo: 520 Files: fields in 48 plan files at rounds 7-8 (round 7
* wrote "182 plans" here; round 8's count of the files is 48), and 560 fields
* in 48 plan files at round 9 (2026-10-11), 548 of them with a backticked
* path. In 3 of those a plain path-like word stands beside it, each inside a
* description, and none of them is read as a file:
*
* - A field with a backticked path names its backticked spans that pass the
* path test (a `.`, `/` or `\`; no URL, option or call like `parse()`),
@ -297,15 +301,19 @@ export function onFailureAction(text) {
* file (`Makefile`, `Dockerfile`), and a backticked capitalised name with no
* extension that stands alone in its entry. Beside them a plain word counts
* only when it is clearly a file or a directory, and only before the entry's
* description starts (a dash, or a colon after the backticked path): a path
* with a slash that ends in a file or a `/` anywhere there ("the CLI entry
* src/cli.mjs"), and a name with an extension, a dotfile or a build file only
* where an entry or a list starts: first in its entry, or after "and", "or",
* "+" or "&" (", Dockerfile", "and Makefile", ", package.json"). Round 7: they
* were dropped, and a step that builds something read as tests-only. Round 8:
* "— mocks fs.readFileSync", "runs on Node.js 22.x" or "no change to
* package.json" read as files, and a step that holds only its test did not.
* "covers pass/fail", "JSON, YAML" or "v6.3.0 format" is no path.
* description starts (a dash, or a colon after a path, backticked or plain:
* "src/cli.mjs: wires parse"): a path with a slash that ends in a file or a
* `/` anywhere there ("the CLI entry src/cli.mjs"), and a name with an
* extension, a dotfile or a build file only where an entry or a list starts:
* first in its entry, or after "and", "or", "+" or "&" (", Dockerfile", "and
* Makefile", ", package.json"). A description runs on past a comma and onto
* the next line until a new entry starts: a list item on a new line, or a
* backticked path. Round 7: such files were dropped, and a step that builds
* something read as tests-only. Rounds 8-9: "— mocks fs.readFileSync", "runs
* on Node.js 22.x", "no change to package.json", "— mocks fs,
* child_process.spawnSync" or "src/cli.mjs: wires parse and README.md" read
* as files, and a step that holds only its test did not. "covers
* pass/fail", "JSON, YAML" or "v6.3.0 format" is no path.
* - A field with no backticked path names every path-like word: a name with an
* extension, a dotfile, a path with a slash, a directory (`docs/`), a glob
* (`src/*.mjs`) or a build file; a parenthesised note is not read. Another
@ -329,6 +337,7 @@ const BUILD_FILE = /^(?!Profile$)(?:[A-Z][a-z]+|GNUmake)file$|^(?:make|rake|just
// Where an entry's description starts, and the words that start a list item.
const DESCRIPTION_DASH = /^(?:\u2014|\u2013|-|--)$/;
const LIST_JOINER = /^(?:and|or|\+|&)$/i;
const LIST_ITEM = /^\s*(?:[-*+]|\d+\.)\s/;
const lastSegment = (p) => p.replace(/\/$/, '').split('/').pop();
const norm = (p) => p.replace(/\\/g, '/').replace(/^\.\//, '');
// A backticked span that is a path, not a URL, an option or a call (`parse()`, `p.parse()`).
@ -347,36 +356,50 @@ export function fieldPaths(text) {
const isTicked = spans.map((p) => tickedPath(p) || BUILD_FILE.test(p));
const tickedField = isTicked.some(Boolean);
const out = [];
for (const raw of masked.split(/[,\n]/)) {
const entry = raw.trim().replace(/^[-*]\s+/, '');
if (entry === '') continue;
const tokens = entry.split(/\s+/);
// A backticked capitalised name alone in its entry is a file (round 6).
const alone = /^\u0000(\d+)\u0000$/.exec(entry);
if (alone && BARE_NAME.test(spans[Number(alone[1])])) { out.push(spans[Number(alone[1])]); continue; }
if (tickedField) {
// In the order they stand: the backticked paths, and before the
// description a plain word that is clearly a file where it may stand
// (rounds 7-8).
let described = false;
for (let i = 0; i < tokens.length; i++) {
const t = tokens[i];
if (t.includes('\u0000')) {
out.push(...[...t.matchAll(/\u0000(\d+)\u0000/g)].map((m) => Number(m[1])).filter((n) => isTicked[n]).map((n) => spans[n]));
if (t.endsWith(':')) described = true;
continue;
}
if (DESCRIPTION_DASH.test(t)) described = true;
const w = plainWord(t);
if (described || !isFileWord(w)) continue;
if (w.includes('/') || i === 0 || LIST_JOINER.test(tokens[i - 1])) out.push(w);
// A description runs on past a comma and onto the next line until a new
// entry starts: a list item on a new line, or a backticked path (round 9).
let described = false;
for (const line of masked.split('\n')) {
if (LIST_ITEM.test(line)) described = false;
for (const raw of line.split(',')) {
const entry = raw.trim().replace(/^[-*]\s+/, '');
if (entry === '') continue;
const tokens = entry.split(/\s+/);
const first = /^\u0000(\d+)\u0000/.exec(entry);
if (first && isTicked[Number(first[1])]) described = false;
// A backticked capitalised name alone in its entry is a file (round 6).
if (first && first[0] === entry && BARE_NAME.test(spans[Number(first[1])])) {
out.push(spans[Number(first[1])]);
described = false;
continue;
}
continue;
if (tickedField) {
// In the order they stand: the backticked paths, and before the
// description a plain word that is clearly a file where it may stand
// (rounds 7-8). A path followed by a colon starts its description,
// backticked or plain (round 9).
for (let i = 0; i < tokens.length; i++) {
const t = tokens[i];
if (t.includes('\u0000')) {
out.push(...[...t.matchAll(/\u0000(\d+)\u0000/g)].map((m) => Number(m[1])).filter((n) => isTicked[n]).map((n) => spans[n]));
if (t.endsWith(':')) described = true;
continue;
}
if (DESCRIPTION_DASH.test(t)) described = true;
const w = plainWord(t);
if (described || !isFileWord(w)) continue;
if (w.includes('/') || i === 0 || LIST_JOINER.test(tokens[i - 1])) {
out.push(w);
if (t.endsWith(':')) described = true;
}
}
continue;
}
const words = tokens.filter((w) => !w.includes('\u0000')).map(plainWord).filter(Boolean);
const found = words.filter((w) => isPathWord(w) || BUILD_FILE.test(w));
if (found.length === 0 && words.length === 1 && tokens.length === 1 && BARE_NAME.test(words[0])) found.push(words[0]);
out.push(...found);
}
const words = tokens.filter((w) => !w.includes('\u0000')).map(plainWord).filter(Boolean);
const found = words.filter((w) => isPathWord(w) || BUILD_FILE.test(w));
if (found.length === 0 && words.length === 1 && tokens.length === 1 && BARE_NAME.test(words[0])) found.push(words[0]);
out.push(...found);
}
return out.filter(Boolean).map(norm);
}

View file

@ -36,33 +36,67 @@ export const FINDING_REQUIRED_FIELDS = Object.freeze([
'line',
]);
// Last fenced ```json … ``` block in a reviewer's output. The contract pins the
// JSON block as the LAST fence so prose above it never confuses the parser.
// Both fences stand on a line of their own, as in Markdown and in the verdict
// gate's reader: a JSON string cannot hold a line break, so three backticks
// inside one ("a fenced ```bash block") never end the block. Ending it at the
// first triple backtick anywhere made fill-finding-ids exit 2 (release review
// of 6.3.0, found while writing round 7's review.md; present in 6.2.0). As in
// Markdown, the closing fence may be longer than three backticks (round 8);
// three backticks after text on the same line are no fence, here or in the gate.
// One reader for every trailing json block: the review validator and the
// synthesis digest import it.
const JSON_FENCE_GLOBAL = /^[ \t]*```json[ \t]*\r?\n([\s\S]*?)^[ \t]*`{3,}[ \t]*\r?$/gim;
// The fenced blocks of a Markdown text, in order, read as Markdown reads a
// backtick fence: an opening line of three or more backticks whose info string
// holds no backtick, closed only by a line of backticks at least as long as the
// opening run (round 9: any three closed any opener, so a four-backtick example
// of a json block split there). A block never closed runs to the end of the
// text (`closed: false`). Indented or not, a fence counts, also inside a list
// item; a `\r` before a line break is no part of the line, so a CRLF file reads
// as its LF twin. A JSON string cannot hold a line break, so three backticks
// inside one ("a fenced ```bash block") never end a block, and three backticks
// after text on the same line are no fence (release review of 6.3.0, rounds
// 7-9). `info` is the info string as written, trailing blanks dropped; `body`
// is the text between the fence lines, as written.
// One reader for the verdict gate and every trailing-json reader: the review
// validator, the synthesis digest, fill-finding-ids, recurrence, fidelity-diff.
const FENCE_OPEN = /^[ \t]*(`{3,})([^`]*)$/;
const FENCE_CLOSE = /^[ \t]*(`{3,})[ \t]*$/;
/**
* Extract the inner body of the LAST fenced `json` block in `text`.
* @param {string} text
* @returns {Array<{info: string, body: string, closed: boolean}>}
*/
export function fencedBlocks(text) {
const blocks = [];
if (typeof text !== 'string') return blocks;
let open = null;
for (let at = 0; at <= text.length;) {
const nl = text.indexOf('\n', at);
const end = nl === -1 ? text.length : nl;
const line = text.slice(at, end).replace(/\r$/, '');
if (open === null) {
const m = FENCE_OPEN.exec(line);
if (m) open = { run: m[1].length, info: m[2].replace(/[ \t]+$/, ''), start: end + 1 };
} else {
const m = FENCE_CLOSE.exec(line);
if (m && m[1].length >= open.run) {
blocks.push({ info: open.info, body: text.slice(open.start, at), closed: true });
open = null;
}
}
if (nl === -1) break;
at = nl + 1;
}
if (open !== null) blocks.push({ info: open.info, body: text.slice(open.start), closed: false });
return blocks;
}
/** A `json` block: its info string is exactly `json` (any case). */
export function isJsonBlock(block) {
return block.info.toLowerCase() === 'json';
}
/**
* Extract the inner body of the LAST closed `json` block in `text`. The
* contract pins the JSON block as the LAST fence so prose above it never
* confuses the parser.
* @param {string} text
* @returns {string|null} the JSON source, or null if no json fence is present.
*/
export function extractFindingsBlock(text) {
if (typeof text !== 'string') return null;
JSON_FENCE_GLOBAL.lastIndex = 0;
let last = null;
let m;
while ((m = JSON_FENCE_GLOBAL.exec(text)) !== null) {
last = m[1];
}
return last;
const json = fencedBlocks(text).filter((b) => b.closed && isJsonBlock(b));
return json.length > 0 ? json[json.length - 1].body : null;
}
function validateFinding(finding, index, errors) {

View file

@ -28,7 +28,7 @@
import { readFileSync, existsSync } from 'node:fs';
import { validateReviewContent, VERDICT_VALUES } from '../validators/review-validator.mjs';
import { extractFindingsBlock } from './findings-schema.mjs';
import { fencedBlocks, isJsonBlock } from './findings-schema.mjs';
import { isMain } from '../util/is-main.mjs';
export const EXIT = Object.freeze({ PASS: 0, BLOCK: 1, UNDECIDED: 2 });
@ -67,17 +67,23 @@ export function decideVerdict(text) {
if ((fmText.match(/^verdict\s*:/gm) ?? []).length > 1) {
return undecided('frontmatter carries more than one verdict line', verdict);
}
const lastFence = lastFenceLanguage(body);
if (lastFence === null) {
// The last fenced block, its info string and its body from one reading
// (findings-schema `fencedBlocks`): a gate that took the language from one
// reader and the body from another passed a BLOCKER in a last block opened
// as ```json {x} (release review of 6.3.0, round 9).
const last = fencedBlocks(body).at(-1);
if (last === undefined) {
return undecided(`verdict ${verdict} has no trailing JSON block to back it`, verdict);
}
if (lastFence !== 'json') {
return undecided(`the last fenced block is "${lastFence || '(no language)'}", not the trailing \`json\` block`, verdict);
if (!last.closed) {
return undecided(`the last fenced block ("${last.info || '(no language)'}") is never closed, so it runs to the end of the file`, verdict);
}
if (!isJsonBlock(last)) {
return undecided(`the last fenced block is "${last.info || '(no language)'}", not the trailing \`json\` block`, verdict);
}
const block = extractFindingsBlock(body);
let payload;
try {
payload = JSON.parse(block);
payload = JSON.parse(last.body);
} catch (e) {
return undecided(`trailing JSON block does not parse: ${e.message}`, verdict);
}
@ -118,19 +124,6 @@ function atLeastVersion(v, major, minor) {
return Boolean(m) && (Number(m[1]) > major || (Number(m[1]) === major && Number(m[2]) >= minor));
}
// A fence as lib/review/findings-schema.mjs reads one: on a line of its own,
// indented or not, and a closing fence of three backticks or more (round 8).
const FENCE_RE = /^[ \t]*```[ \t]*([A-Za-z0-9_+-]*)[^\n]*\n[\s\S]*?^[ \t]*`{3,}[ \t]*$/gm;
/** Language tag of the LAST fenced block in `body` (lower-cased), '' for none, null when no fence. */
function lastFenceLanguage(body) {
let last = null;
let m;
FENCE_RE.lastIndex = 0;
while ((m = FENCE_RE.exec(body)) !== null) last = m[1].toLowerCase();
return last;
}
/** The JSON block shape the gate needs; returns an error string or null. */
function checkPayloadShape(p) {
if (p === null || typeof p !== 'object' || Array.isArray(p)) return 'not an object';

View file

@ -42,7 +42,7 @@ import { join, resolve, dirname } from 'node:path';
import { createHash } from 'node:crypto';
import { tmpdir } from 'node:os';
import { resolveRepo, canonicalPath } from '../verification/repo-tools.mjs';
import { DEFAULT_TIMEOUT_MS, allowedCommand, dropComment, runCommand, screenCommand, stoppedBy } from '../verification/criteria-runner.mjs';
import { DEFAULT_TIMEOUT_MS, allowedCommand, commandWords, runCommand, screenCommand, stoppedBy } from '../verification/criteria-runner.mjs';
import { isMain } from './is-main.mjs';
export const LOCK_VERSION = 1;
@ -172,8 +172,11 @@ function main(argv) {
} else {
const testCmd = typeof lock.cmd === 'string' && lock.cmd.trim() !== '' ? lock.cmd : null;
if (!testCmd) fail(2, 'the lock names no test command (lock with --cmd "<Verify>"); never marked green');
// One reading of a command passed as written: a trailing comment is no part of it (round 8).
if (args.cmd !== undefined && dropComment(args.cmd) !== dropComment(testCmd)) fail(2, `--cmd is not the step's test command stored in the lock (${testCmd})`);
// One reading of a command passed as written: its words, as runCommand
// runs them, so a trailing comment (round 8) and the blanks between the
// words (round 9) are no part of it.
const words = (c) => commandWords(c).join(' ');
if (args.cmd !== undefined && words(args.cmd) !== words(testCmd)) fail(2, `--cmd is not the step's test command stored in the lock (${testCmd})`);
const allowed = allowedCommand(testCmd, { cwd: cur.repo.root });
if (!allowed.allowed) fail(2, `command not run: ${allowed.reason}`);
const screen = screenCommand(testCmd);

View file

@ -44,7 +44,7 @@ import {
import { validateAllManifests } from '../parsers/manifest-yaml.mjs';
import { parseDocument } from '../util/frontmatter.mjs';
import { issue, fail } from '../util/result.mjs';
import { parsePlanVerification, firstCommand, runnableCheck } from '../verification/criteria-runner.mjs';
import { parsePlanVerification, firstCommand, runnableCheck, refusal } from '../verification/criteria-runner.mjs';
import { isMain } from '../util/is-main.mjs';
export const OPTIONAL_STEP_FIELDS = Object.freeze(['Interface in', 'Interface out', 'Review focus']);
@ -109,7 +109,7 @@ function checkRunnable(text, sections, opts) {
`Phase 7 runs every criterion and fells the run on one it cannot run. ${WAY_IN}`));
}
}
const verified = new Set(criteria.filter(c => c.command).map(c => c.command.trim().replace(/\s+/g, ' ')));
const verified = new Set(criteria.filter(c => !refusal(c)).map(c => c.command.trim().replace(/\s+/g, ' ')));
for (const s of sections) {
const f = parseStepFields(s.body);
if (f.has('Verify')) {

View file

@ -155,6 +155,11 @@ const COMMENT_NOT_PLAIN = 'a trailing comment with a quote, $, backtick or backs
// The first backtick-delimited span on the line is the command by convention —
// both templates put it first and a second span holds the expected output.
// A plan step's `Verify:` and `End-to-end:` fields follow the same convention.
// A `reason` says why the span cannot run. It comes with `command: null` when
// the span names nothing to run, and beside the command when only its comment
// is refused: that span does carry a command, so it stays a criterion of its
// own and is refused out loud (`refusal`), never read as prose or as a
// sub-point (release review of 6.3.0, round 9).
export function firstCommand(text) {
const m = text.match(/`([^`]+)`/);
if (!m) return { command: null, reason: 'no-command' };
@ -169,10 +174,16 @@ export function firstCommand(text) {
// (`--cmd "{Verify command}"`): there a `"` ends the argument and a `$`, a
// backtick or a backslash still means something (release review of 6.3.0,
// round 8). A comment with one of them is refused, never dropped.
if (/["$`\\]/.test(span.slice(raw.length))) return { command: null, reason: COMMENT_NOT_PLAIN };
if (/["$`\\]/.test(span.slice(raw.length))) return { command: raw, reason: COMMENT_NOT_PLAIN };
return { command: raw, reason: '' };
}
/** Why a criterion cannot run (no command, or a refused one), or '' when it may be screened. */
export function refusal(criterion) {
if (!criterion?.command) return criterion?.reason || 'no command';
return criterion.reason || '';
}
/**
* A command without its trailing comment (`npm test # all green`): a shell
* drops it, and with no shell (runCommand) its words would reach the screen
@ -477,7 +488,8 @@ export function allowedCommand(command, opts = {}) {
* what adds it. Returns `{runnable, reason}`.
*/
export function runnableCheck(criterion, opts = {}) {
if (!criterion?.command) return { runnable: false, reason: criterion?.reason || 'no command' };
const refused = refusal(criterion);
if (refused) return { runnable: false, reason: refused };
const r = screenShape(criterion.command, opts, true);
return { runnable: r.allowed, reason: r.reason };
}
@ -596,8 +608,9 @@ export function runCriteria(criteria, opts = {}) {
return criteria.map((c) => {
const base = { label: c.label, text: c.text, command: c.command };
if (!c.command) {
return { ...base, status: 'unrunnable', exitCode: null, output: `not runnable: ${c.reason || 'no command'}` };
const refused = refusal(c);
if (refused) {
return { ...base, status: 'unrunnable', exitCode: null, output: `not runnable: ${refused}` };
}
const permission = allow(c.command, { cwd });
if (!permission.allowed) {
@ -649,13 +662,14 @@ export function summarize(results, opts = {}) {
return summary;
}
/** The commands a plan's steps declare in `End-to-end:` (first backticked span each). */
/** The commands a plan's steps declare in `End-to-end:` (first backticked span each; a refused one is none). */
export function endToEndCommands(markdown) {
return sliceSteps(String(markdown ?? '').replace(/\r\n?/g, '\n'))
.map((s) => parseStepFields(s.body).get('End-to-end'))
.filter(Boolean)
.map((f) => firstCommand(f.text).command)
.filter(Boolean);
.map((f) => firstCommand(f.text))
.filter((c) => !refusal(c))
.map((c) => c.command);
}
const normalizeCommand = (c) => String(c).trim().replace(/\s+/g, ' ');