fix: round 5 of the release review of 6.3.0, the structural fix
Round 5 (v6.2.0..d37e538, WARN: 11 MAJOR, 2 MINOR). The patching did not converge (the same two places every round), so the operator chose a structural fix on 2026-10-11. Each change behind a test that failed first. - A check runs with no shell. criteria-runner's runCommand starts the runner with the very words the screen checked (spawnSync(words[0], words.slice(1)), without NODE_TEST_CONTEXT); Phase 7 criteria, Regression: (its exec copy is gone, MINOR 0e98d5c7), proof of red, the done line and the test lock all run through it. No quote, glob, tilde or variable can rewrite a checked word any more. The screen stays and says why: a word that means something else in a shell (quote, backslash, $, brace, leading ~) is refused; a glob reaches only node --test, which expands its own patterns (sh read `tests/.?/` as `tests/../`); no path climbs out with .. or starts at /; a word starting with @ is refused (pytest reads @file as more arguments). - The K6 plan-file sweep has v6.2.0's own Files: reader again (516dffa): backticked paths, and plain words only in a field without them. The shared reader from round 3 planned directories a description named and dropped paths v6.2.0 planned. fieldPaths stays for the tests-only check, with a path test on spans (`parse()`, `p.parse()` are no paths), plain descriptions ignored when the field has backticked paths, dotfiles, backslash paths and glued spans read. - Skill frontmatter reads a value as YAML does: trailing space is nothing, and a comment ends a plain value, a flow list or a block indicator (`model: opus # pinned`). A comment that cuts a plain description, or is the whole value, is still a hazard. - release-check reads the diff with --no-renames: a test moved into an allowed place after the review shows as removed. - The example plan's step 2 heading no longer says "must FAIL initially"; steps 1-2 keep the tests-only warning (operator 2026-10-09,73262b6). Open: MINOR c1fef1ad (one shared isMain helper) is not in this round. Measured: 0 of 463 Files: fields with backticked paths (520 fields in 182 plans across the marketplace repos) carry another path-like word. Suite 2185 (2183/0/2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
d37e538ddc
commit
c3641ffa42
SSH key fingerprint:
SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
17 changed files with 409 additions and 111 deletions
|
|
@ -285,48 +285,65 @@ export function onFailureAction(text) {
|
|||
// ---- a step that holds nothing but tests -----------------------------------
|
||||
|
||||
/**
|
||||
* The paths a field names, in order. A backticked span is a path as written,
|
||||
* parentheses and all (`app/(auth)/x.sh`), also with punctuation after it
|
||||
* (`a.mjs`.) and inside a note ("(fixture: `t/a.json`)"). In the plain text a
|
||||
* parenthesised note with no span in it is not a path ("(new: covers x, y)"),
|
||||
* and every word that looks like a path is one: a name with an extension, a
|
||||
* path with a slash, a directory (`docs/`) or a glob (`src/*.mjs`). A word
|
||||
* with no slash and no extension counts only alone in its entry and
|
||||
* capitalised, like `Makefile`; "format" or "v6.3.0" in a description is no
|
||||
* path. Release review of 6.3.0, rounds 1-4.
|
||||
* The paths a step's `Files:` field names, in order, for the tests-only check
|
||||
* below (the K6 sweep keeps v6.2.0's own reader: plan-scope-sweep.mjs,
|
||||
* operator 2026-10-11). The convention in the plan template is a backticked
|
||||
* path, and in every plan measured (520 Files: fields in 182 plans, 11.10)
|
||||
* a field with backticked paths had no other path-like word beside them:
|
||||
*
|
||||
* - A field with a backticked path names its backticked spans that pass the
|
||||
* path test (a `.`, `/` or `\`; no URL, option or call like `parse()`),
|
||||
* wherever they sit (`a.mjs`/`b.mjs`, inside a note). Its descriptions are
|
||||
* not read: "covers pass/fail", "JSON, YAML" or a directory named in passing
|
||||
* is no path. An entry of its own with no backticks (", .gitignore") counts
|
||||
* by its first word, and only when that is clearly a file or a directory.
|
||||
* - A field with no backticked path names every path-like word: a name with an
|
||||
* extension, a dotfile, a path with a slash, a directory (`docs/`) or a glob
|
||||
* (`src/*.mjs`); a parenthesised note is not read. A word with no slash and
|
||||
* no extension counts only alone in its entry and capitalised (`Makefile`);
|
||||
* "format" or "v6.3.0" in a description is no path.
|
||||
*
|
||||
* A backslash is read as a slash, a leading `./` is dropped. Release review of
|
||||
* 6.3.0, rounds 1-5.
|
||||
*/
|
||||
const SEG = '[\\w@.*?()-]';
|
||||
const PATH_WORD = new RegExp(`^(?:${SEG}+\\/)+${SEG}*$|^[\\w@*?-][\\w@.*?-]*\\.[A-Za-z*][A-Za-z0-9*]*$`);
|
||||
const PATH_WORD = new RegExp(`^(?:${SEG}+\\/)+${SEG}*$|^[\\w@*?-][\\w@.*?-]*\\.[A-Za-z*][A-Za-z0-9*]*$|^\\.[\\w-][\\w.-]*$`);
|
||||
// Abbreviations that PATH_WORD would read as a name with an extension.
|
||||
const NOT_A_PATH = /^(?:e\.g|i\.e|n\.b|cf|etc|vs)$/i;
|
||||
const BARE_NAME = /^[A-Z][\w.-]*$/;
|
||||
const dotSlash = (p) => p.replace(/^\.\//, '');
|
||||
/**
|
||||
* The one reader of a step's `Files:` field: this file's tests-only check and
|
||||
* the plan-file sweep (lib/verification/plan-scope-sweep.mjs) both use it, so
|
||||
* the two never judge different lists for one step. A leading `./` is dropped.
|
||||
*/
|
||||
// The last segment is a dotfile or a name with an extension, or the path is a directory.
|
||||
const FILE_OR_DIR = /(?:^|\/)(?:\.[\w-][\w.-]*|[^/]*\.[A-Za-z*][A-Za-z0-9*]*)$|\/$/;
|
||||
const norm = (p) => p.replace(/\\/g, '/').replace(/^\.\//, '');
|
||||
// A backticked span that is a path, not a URL, an option or a call (`parse()`, `p.parse()`).
|
||||
const tickedPath = (s) => /[./\\]/.test(s) && !/^https?:/.test(s) && !s.startsWith('-') && !/^[\w$.]+\(.*\)$/.test(s);
|
||||
// A plain word without the brackets and punctuation around it, `\` read as `/`.
|
||||
const plainWord = (w) => w.replace(/^[(\[]+/, '').replace(/[.,;:)\]]+$/, '').replace(/\\/g, '/');
|
||||
const isPathWord = (w) => PATH_WORD.test(w) && !NOT_A_PATH.test(w);
|
||||
|
||||
export function fieldPaths(text) {
|
||||
const spans = [];
|
||||
const masked = String(text ?? '')
|
||||
.replace(/`([^`\n]+)`/g, (_, p) => `\u0000${spans.push(p.trim()) - 1}\u0000`)
|
||||
.replace(/(^|\s)\(([^()\n]*)\)/g, (m, pre, inner) => (inner.includes('\u0000') ? m : `${pre} `));
|
||||
const isTicked = spans.map(tickedPath);
|
||||
const tickedField = isTicked.some(Boolean);
|
||||
const out = [];
|
||||
for (const raw of masked.split(/[,\n]/)) {
|
||||
const entry = raw.trim().replace(/^[-*]\s+/, '');
|
||||
if (entry === '') continue;
|
||||
const words = entry.split(/\s+/);
|
||||
const found = [];
|
||||
for (const word of words) {
|
||||
const w = word.replace(/^[(\[]+/, '').replace(/[.,;:)\]]+$/, '');
|
||||
const span = /^\u0000(\d+)\u0000$/.exec(w);
|
||||
if (span) found.push(spans[Number(span[1])]);
|
||||
else if (PATH_WORD.test(w) && !NOT_A_PATH.test(w)) found.push(w);
|
||||
const ticked = [...entry.matchAll(/\u0000(\d+)\u0000/g)].map((m) => Number(m[1])).filter((i) => isTicked[i]);
|
||||
if (ticked.length) { out.push(...ticked.map((i) => spans[i])); continue; }
|
||||
const words = entry.split(/\s+/).filter((w) => !w.includes('\u0000')).map(plainWord).filter(Boolean);
|
||||
if (tickedField) {
|
||||
const first = plainWord(entry.split(/\s+/)[0]);
|
||||
if (!first.includes('\u0000') && isPathWord(first) && FILE_OR_DIR.test(first)) out.push(first);
|
||||
continue;
|
||||
}
|
||||
if (found.length === 0 && words.length === 1 && BARE_NAME.test(words[0])) found.push(words[0]);
|
||||
const found = words.filter(isPathWord);
|
||||
if (found.length === 0 && words.length === 1 && entry.split(/\s+/).length === 1 && BARE_NAME.test(words[0])) found.push(words[0]);
|
||||
out.push(...found);
|
||||
}
|
||||
return out.filter((p) => p && !p.includes('\u0000')).map(dotSlash);
|
||||
return out.filter(Boolean).map(norm);
|
||||
}
|
||||
|
||||
// A document or a data file outside tests/ is never the test here:
|
||||
|
|
@ -358,7 +375,7 @@ export function stepTestsOnly(body) {
|
|||
export function stepTestsOnlyFromFields(f) {
|
||||
const files = f.has('Files') ? fieldPaths(f.get('Files').text) : [];
|
||||
const testFirst = f.has('Test first')
|
||||
? [...f.get('Test first').text.matchAll(/\bFile:\**[ \t]*`([^`\n]+)`/gi)].map((m) => dotSlash(m[1].trim()))
|
||||
? [...f.get('Test first').text.matchAll(/\bFile:\**[ \t]*`([^`\n]+)`/gi)].map((m) => norm(m[1].trim()))
|
||||
: [];
|
||||
const isTest = (p) => testFirst.includes(p) ||
|
||||
(isTestPath(p) && (UNDER_TESTS_RE.test(p) || !(DOC_RE.test(p) || DATA_RE.test(p))));
|
||||
|
|
|
|||
|
|
@ -10,6 +10,14 @@
|
|||
// real one. Claude Code loads a skill whose YAML does not parse "with no
|
||||
// fields set", so the description is gone and the skill cannot match; a trap
|
||||
// here is an error, not a style note.
|
||||
//
|
||||
// A value is read as YAML reads it: trailing space on the line is nothing, and
|
||||
// a comment (` #`, or `#` at the start) ends a plain value, a flow list or a
|
||||
// block indicator (`model: opus # pinned` is `opus`). Two comments are still
|
||||
// traps: one that is the whole value (the value is empty), and one that cuts a
|
||||
// plain `description`, the text Claude Code routes on (`Use when issue #42
|
||||
// …` would route on "Use when issue"); quote the description if "#" belongs to
|
||||
// it. Release review of 6.3.0, rounds 3-5.
|
||||
|
||||
const KEY_RE = /^([A-Za-z0-9_-]+):(.*)$/;
|
||||
const BLOCK_RE = /^([|>])([+-]?)\s*$/;
|
||||
|
|
@ -27,7 +35,7 @@ export function splitFile(text) {
|
|||
return { ok: true, fmLines: lines.slice(1, end), body: lines.slice(end + 1).join('\n'), bodyStartLine: end + 2 };
|
||||
}
|
||||
|
||||
/** The YAML traps in one plain (unquoted, single-line) scalar value. */
|
||||
/** The YAML traps in one plain (unquoted, single-line) scalar value, meant whole. */
|
||||
export function plainScalarHazards(value) {
|
||||
const out = [];
|
||||
if (/:\s/.test(value) || /:$/.test(value)) out.push('unquoted ": " (YAML reads a mapping and the line fails)');
|
||||
|
|
@ -39,6 +47,24 @@ export function plainScalarHazards(value) {
|
|||
return out;
|
||||
}
|
||||
|
||||
// Where a YAML comment starts in a value: `#` at the start or after a space or
|
||||
// tab, outside a quoted item (a quote opens only at the start of a token, so
|
||||
// the apostrophe in `don't` is text). -1 when there is none.
|
||||
function commentAt(v) {
|
||||
let q = null;
|
||||
for (let i = 0; i < v.length; i++) {
|
||||
const c = v[i];
|
||||
if (q) {
|
||||
if (q === '"' && c === '\\') i++;
|
||||
else if (c === q) q = null;
|
||||
continue;
|
||||
}
|
||||
if ((c === '"' || c === "'") && (i === 0 || /[\s,[]/.test(v[i - 1]))) { q = c; continue; }
|
||||
if (c === '#' && (i === 0 || v[i - 1] === ' ' || v[i - 1] === '\t')) return i;
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
// A quoted value ends at its closing quote; after it only a YAML comment may
|
||||
// follow (`name: "x" # pinned`). Inside double quotes a backslash escapes the
|
||||
// next character; inside single quotes '' is one quote.
|
||||
|
|
@ -101,7 +127,7 @@ export function readFrontmatterFields(text) {
|
|||
const m = KEY_RE.exec(line);
|
||||
if (!m) { hazards.push({ line: lineNo, message: `not a "key: value" line: ${line.slice(0, 60)}` }); i++; continue; }
|
||||
const key = m[1];
|
||||
const rest = m[2].replace(/^[ \t]+/, '');
|
||||
const rest = m[2].trim();
|
||||
if (m[2] !== '' && !/^[ \t]/.test(m[2])) hazards.push({ line: lineNo, message: `"${key}:" needs a space after the colon` });
|
||||
if (key in fields) hazards.push({ line: lineNo, message: `duplicate key "${key}"` });
|
||||
|
||||
|
|
@ -111,16 +137,23 @@ export function readFrontmatterFields(text) {
|
|||
while (j < fmLines.length && (fmLines[j].trim() === '' || /^\s/.test(fmLines[j]))) { cont.push(fmLines[j]); j++; }
|
||||
while (cont.length && cont[cont.length - 1].trim() === '') { cont.pop(); }
|
||||
|
||||
const block = BLOCK_RE.exec(rest);
|
||||
// A quoted value keeps its own comment rule (unquote); any other value
|
||||
// ends where a comment starts.
|
||||
const quoted = rest[0] === '"' || rest[0] === "'";
|
||||
const cut = quoted ? -1 : commentAt(rest);
|
||||
const value = cut < 0 ? rest : rest.slice(0, cut).trimEnd();
|
||||
|
||||
const block = BLOCK_RE.exec(value);
|
||||
if (block) {
|
||||
fields[key] = { value: blockValue(cont, block[1], block[2]), line: lineNo, style: 'block' };
|
||||
} else if (rest === '') {
|
||||
} else if (value === '') {
|
||||
const items = cont.filter((l) => l.trim() !== '');
|
||||
if (items.length && items.every((l) => /^\s+-\s/.test(l))) {
|
||||
fields[key] = { value: items.map((l) => l.replace(/^\s+-\s+/, '').trim()), line: lineNo, style: 'list' };
|
||||
} else {
|
||||
fields[key] = { value: items.join('\n'), line: lineNo, style: items.length ? 'map' : 'empty' };
|
||||
}
|
||||
if (cut === 0 && !items.length) hazards.push({ line: lineNo, message: `${key}: "#" at the start is a YAML comment: the value is empty` });
|
||||
} else {
|
||||
if (cont.length) hazards.push({ line: lineNo, message: `"${key}" has a value and indented lines under it` });
|
||||
const q = unquote(rest);
|
||||
|
|
@ -129,15 +162,18 @@ export function readFrontmatterFields(text) {
|
|||
fields[key] = { value: rest, line: lineNo, style: 'broken' };
|
||||
} else if (q) {
|
||||
fields[key] = { value: q.value, line: lineNo, style: 'quoted' };
|
||||
} else if (/^\[.*\]$/.test(rest)) {
|
||||
const inner = rest.slice(1, -1).trim();
|
||||
} else if (/^\[.*\]$/.test(value)) {
|
||||
const inner = value.slice(1, -1).trim();
|
||||
fields[key] = {
|
||||
value: inner === '' ? [] : inner.split(',').map((s) => { const t = s.trim(); const u = unquote(t); return u && !u.error ? u.value : t; }),
|
||||
line: lineNo, style: 'flow',
|
||||
};
|
||||
} else {
|
||||
for (const h of plainScalarHazards(rest)) hazards.push({ line: lineNo, message: `${key}: ${h}` });
|
||||
fields[key] = { value: rest, line: lineNo, style: 'plain' };
|
||||
for (const h of plainScalarHazards(value)) hazards.push({ line: lineNo, message: `${key}: ${h}` });
|
||||
if (cut > 0 && key === 'description') {
|
||||
hazards.push({ line: lineNo, message: `${key}: " #" starts a YAML comment: the description is cut to "${value}"; quote it if "#" belongs to it` });
|
||||
}
|
||||
fields[key] = { value, line: lineNo, style: 'plain' };
|
||||
}
|
||||
}
|
||||
i = j;
|
||||
|
|
|
|||
|
|
@ -42,10 +42,9 @@ import { join, resolve, dirname } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { resolveRepo, canonicalPath } from '../verification/repo-tools.mjs';
|
||||
import { allowedCommand, screenCommand } from '../verification/criteria-runner.mjs';
|
||||
import { commandEnv, DEFAULT_TIMEOUT_MS } from '../verification/red-proof.mjs';
|
||||
import { allowedCommand, runCommand, screenCommand } from '../verification/criteria-runner.mjs';
|
||||
import { DEFAULT_TIMEOUT_MS } from '../verification/red-proof.mjs';
|
||||
|
||||
export const LOCK_VERSION = 1;
|
||||
|
||||
|
|
@ -179,9 +178,8 @@ function main(argv) {
|
|||
if (!allowed.allowed) fail(2, `command not run: ${allowed.reason}`);
|
||||
const screen = screenCommand(testCmd);
|
||||
if (!screen.allowed) fail(2, `command blocked: ${screen.rule}`);
|
||||
const r = spawnSync('/bin/sh', ['-c', testCmd], {
|
||||
cwd: cur.repo.root, env: commandEnv(), encoding: 'utf8', timeout: DEFAULT_TIMEOUT_MS, maxBuffer: 64 * 1024 * 1024,
|
||||
});
|
||||
// No shell: the words the screen checked are the words that run.
|
||||
const r = runCommand(testCmd, { cwd: cur.repo.root, timeoutMs: DEFAULT_TIMEOUT_MS });
|
||||
if (r.error || r.status === null || r.status === 126 || r.status === 127) {
|
||||
fail(2, `the test command did not run to an exit code (${r.error?.message ?? r.signal ?? `exit ${r.status}`})`);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -21,7 +21,17 @@
|
|||
// tests/<script>.sh`, a read-only git subcommand, …) — anything else is NOT
|
||||
// RUN, never a failure — and what survives is then screened through the
|
||||
// plugin's own PreToolUse denylist (hooks/scripts/pre-bash-executor.mjs) —
|
||||
// catastrophe — before it reaches a shell.
|
||||
// catastrophe — before it runs.
|
||||
//
|
||||
// No shell. A command runs as its words (`runCommand`): the runner is started
|
||||
// directly with the words the screen checked, so no quoting, glob, tilde,
|
||||
// brace or variable can turn them into other words after the check. Five
|
||||
// review rounds of 6.3.0 found one more way /bin/sh rewrote a checked word
|
||||
// (quotes, tilde, a glob that reached `..`); with no shell there is nothing
|
||||
// left to rewrite (operator 2026-10-11). The same function runs every check
|
||||
// Voyage runs: Phase 7 criteria, `Regression:` (regression-check.mjs), proof
|
||||
// of red (red-proof.mjs), the done line (done-line.mjs), the test lock
|
||||
// (lib/util/test-lock.mjs).
|
||||
// Commands spawned from here do not pass through the Bash tool, so that hook
|
||||
// would not otherwise fire; the hook is invoked over its documented stdin
|
||||
// protocol rather than copied, so there is exactly one denylist.
|
||||
|
|
@ -257,13 +267,21 @@ export function screenCommand(command, opts = {}) {
|
|||
// - It is not a sandbox. `npm test`, `npm run <script>` and `make test` run
|
||||
// whatever the repo's own package.json/Makefile says they run, including a
|
||||
// script that pushes. That is the repo's responsibility, not this file's.
|
||||
// - It is not an option parser. Each runner takes only plain words (no
|
||||
// quote, backslash, `$` or brace; a glob only inside a path) and a short
|
||||
// list of options that only read (RUNNER_ARGS); an option that writes a
|
||||
// file or loads code (`--test-reporter-destination`, `--outputFile`,
|
||||
// `--junitxml`, `--basetemp`, `--import`, `--config`, `-o`, `-exec`) is on
|
||||
// no list (release review of 6.3.0, round 3: rounds 1-2 closed this for
|
||||
// git only).
|
||||
// - It is not an option parser. Each runner takes only plain words and a
|
||||
// short list of options that only read (RUNNER_ARGS); an option that
|
||||
// writes a file or loads code (`--test-reporter-destination`,
|
||||
// `--outputFile`, `--junitxml`, `--basetemp`, `--import`, `--config`,
|
||||
// `-o`, `-exec`) is on no list (release review of 6.3.0, round 3: rounds
|
||||
// 1-2 closed this for git only).
|
||||
// - Plain words: the words reach the runner as written, so a word that
|
||||
// would mean something else in a shell — a quote, backslash, `$`, brace,
|
||||
// a leading `~` — is refused rather than passed on with a meaning its
|
||||
// author did not intend. A glob is passed on only to `node --test`, which
|
||||
// expands its own patterns (Node 21 and later); no other runner here
|
||||
// expands one, so elsewhere it is refused. A path that climbs out with
|
||||
// `..` or starts at `/` is refused: a check runs in the repo. A word that
|
||||
// starts with `@` is refused: pytest (8.2 and later) reads `@file` as a
|
||||
// file of more arguments, options the list would refuse.
|
||||
// - It rejects plenty of honest commands: an env prefix (`CI=1 npm test`), a
|
||||
// project's own binary (`./build/app --check`), anything piped. A check
|
||||
// that needs one of those is declared through a runner instead —
|
||||
|
|
@ -274,7 +292,8 @@ const SHELL_OPERATORS = /[;&|<>]|\$\(|`/;
|
|||
|
||||
// Read-only git subcommands: they report, they do not write a repository.
|
||||
const READONLY_GIT = new Set(['status', 'log', 'diff', 'show', 'ls-files']);
|
||||
// A word /bin/sh hands to git unchanged (refs like HEAD~1 and HEAD^, formats like %h, paths).
|
||||
// A word that means the same to git with or without a shell (refs like HEAD~1
|
||||
// and HEAD^, formats like %h, paths).
|
||||
const GIT_LITERAL_WORD = /^[A-Za-z0-9_@%^~:.,=+/-]+$/;
|
||||
// `--output`, `--output=<file>`, or an abbreviation of it git may accept.
|
||||
function writesOutput(word) {
|
||||
|
|
@ -285,10 +304,14 @@ function writesOutput(word) {
|
|||
const deny = (reason) => ({ allowed: false, reason });
|
||||
const permit = () => ({ allowed: true, reason: '' });
|
||||
|
||||
// A word /bin/sh hands on unchanged, or a path with a glob in it: a glob can
|
||||
// only expand to names that start with the word's first character, so it
|
||||
// cannot make an option out of a path.
|
||||
const PLAIN_WORD = /^(?:[A-Za-z0-9_@%^~:.,=+/-]+|[A-Za-z0-9_./][A-Za-z0-9_@%^~:.,=+/*?-]*)$/;
|
||||
// A word that means the same with or without a shell.
|
||||
const PLAIN_WORD = /^[A-Za-z0-9_@%^~:.,=+/-]+$/;
|
||||
// The same, with a glob in it: passed on only to `node --test`, which expands
|
||||
// its own patterns (sh would have expanded `.?` to `..`).
|
||||
const GLOB_WORD = /^[A-Za-z0-9_./][A-Za-z0-9_@%^~:.,=+/*?-]*$/;
|
||||
const takesGlobs = (t) => t[0] === 'node' && t[1] === '--test';
|
||||
// A path outside the repo: one that climbs with `..` or starts at `/`.
|
||||
const leavesRepo = (w) => w.startsWith('/') || w.split('/').includes('..');
|
||||
|
||||
// After each runner's fixed words: the options it may take (`name` or
|
||||
// `name=value`; `value` lists the ones whose value may be the next word, and
|
||||
|
|
@ -386,14 +409,14 @@ const RUNNERS = {
|
|||
'/bin/bash': (t) => (isTestScript(t[1]) ? permit() : deny('/bin/bash, but not a script under tests/')),
|
||||
git: (t) => {
|
||||
if (!READONLY_GIT.has(t[1])) return deny(`git, but not a read-only subcommand first (${[...READONLY_GIT].join('|')})`);
|
||||
// The command runs through /bin/sh, which unquotes and expands what it is
|
||||
// given, so a check on the words here holds only for words the shell passes
|
||||
// through unchanged: no quote, backslash, `$`, glob or brace. Then
|
||||
// `--output` (and an abbreviation of it) is refused: it makes diff/log/show
|
||||
// write to a file (release review of 6.3.0, 2026-10-10: a plan's check
|
||||
// could overwrite any file, first plainly, then through quoting).
|
||||
// The words reach git as written (no shell), and only words that mean the
|
||||
// same with or without one run: no quote, backslash, `$`, glob or brace.
|
||||
// Then `--output` (and an abbreviation of it) is refused: it makes
|
||||
// diff/log/show write to a file (release review of 6.3.0, 2026-10-10: a
|
||||
// plan's check could overwrite any file, first plainly, then through
|
||||
// quoting while a shell still ran it).
|
||||
const odd = t.find((w) => !GIT_LITERAL_WORD.test(w));
|
||||
if (odd !== undefined) return deny(`git with a word the shell would rewrite (${odd}): only plain words are checked`);
|
||||
if (odd !== undefined) return deny(`git with a word that means something else in a shell (${odd}): only plain words run, as written`);
|
||||
if (t.some(writesOutput)) return deny('git with --output writes a file, so it is not a read-only check');
|
||||
return permit();
|
||||
},
|
||||
|
|
@ -431,27 +454,54 @@ function screenShape(command, opts, writeTime) {
|
|||
const operator = text.match(SHELL_OPERATORS);
|
||||
if (operator) return deny(`a shell operator (${operator[0]}) — one criterion declares one command, unpiped`);
|
||||
|
||||
const tokens = text.trim().split(/\s+/);
|
||||
const tokens = commandWords(text);
|
||||
if (!Object.hasOwn(RUNNERS, tokens[0])) return deny(`\`${tokens[0]}\` is not a known test runner`);
|
||||
// /bin/sh expands a tilde at the start of a word (and after = or : in an
|
||||
// assignment), so such a word is not one the shell passes on unchanged.
|
||||
// A shell expands a tilde at the start of a word (and after = or : in an
|
||||
// assignment); no shell runs this, so the runner would get the ~ as written.
|
||||
const tilde = tokens.find((w) => /(^|[=:])~/.test(w));
|
||||
if (tilde !== undefined) return deny(`a word the shell would rewrite (${tilde}): a leading ~ is expanded`);
|
||||
if (tilde !== undefined) return deny(`a word that means something else in a shell (${tilde}): a leading ~ is not expanded here`);
|
||||
const form = RUNNERS[tokens[0]](tokens, cwd, writeTime);
|
||||
if (!form.allowed || tokens[0] === 'git') return form;
|
||||
const odd = tokens.find((w) => !PLAIN_WORD.test(w));
|
||||
if (odd !== undefined) return deny(`a word the shell would rewrite (${odd}): only plain words are checked`);
|
||||
const words = tokens.slice(1);
|
||||
const odd = words.find((w) => !PLAIN_WORD.test(w) && !(takesGlobs(tokens) && GLOB_WORD.test(w)));
|
||||
if (odd !== undefined) {
|
||||
return deny(GLOB_WORD.test(odd)
|
||||
? `a glob (${odd}): no shell runs this, and \`${tokens[0]}\` does not expand one itself; name the files, or use \`node --test\``
|
||||
: `a word that means something else in a shell (${odd}): only plain words run, as written`);
|
||||
}
|
||||
const at = words.find((w) => w.startsWith('@'));
|
||||
if (at !== undefined) return deny(`a word that starts with @ (${at}): pytest reads @file as a file of more arguments`);
|
||||
const out = words.find((w) => !w.startsWith('-') && leavesRepo(w));
|
||||
if (out !== undefined) return deny(`a path outside the repo (${out}): a check runs in the repo, with no .. and no absolute path`);
|
||||
return Object.hasOwn(RUNNER_ARGS, tokens[0]) ? screenArgs(tokens, RUNNER_ARGS[tokens[0]]) : permit();
|
||||
}
|
||||
|
||||
/** The words of a command: what the screen checks is exactly what runs. */
|
||||
export function commandWords(command) {
|
||||
return String(command ?? '').trim().split(/\s+/);
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a screened command with no shell: its first word is the program (found
|
||||
* on PATH), the rest are its arguments, as written. Without NODE_TEST_CONTEXT
|
||||
* (lib/util/command-env.mjs): started under `node --test`, a nested `node
|
||||
* --test` reported to the parent and exited 0 on a red test, a false green
|
||||
* (order 10, 2026-10-09). Returns spawnSync's result: `status` is null when
|
||||
* the program could not start (`error`) or was killed (`signal`).
|
||||
*/
|
||||
export function runCommand(command, { cwd, timeoutMs = DEFAULT_TIMEOUT_MS } = {}) {
|
||||
const [program, ...args] = commandWords(command);
|
||||
return spawnSync(program, args, { cwd, env: commandEnv(), encoding: 'utf8', timeout: timeoutMs, maxBuffer: 64 * 1024 * 1024 });
|
||||
}
|
||||
|
||||
// --- running ----------------------------------------------------------------
|
||||
|
||||
// Without NODE_TEST_CONTEXT (lib/util/command-env.mjs): started under
|
||||
// `node --test`, a nested `node --test` criterion reported to the parent and
|
||||
// exited 0 on a red test - a false green (order 10, 2026-10-09).
|
||||
// A program that could not start (not installed: ENOENT) or was killed by the
|
||||
// timeout has no exit code; its error is the output, so the row says why.
|
||||
function defaultExec(command, { cwd, timeoutMs }) {
|
||||
const r = spawnSync('/bin/sh', ['-c', command], { cwd, encoding: 'utf8', timeout: timeoutMs, env: commandEnv() });
|
||||
return { status: r.status, stdout: r.stdout ?? '', stderr: r.stderr ?? '' };
|
||||
const r = runCommand(command, { cwd, timeoutMs });
|
||||
const why = r.error ? `${r.error.message}\n` : r.signal ? `killed (${r.signal})\n` : '';
|
||||
return { status: r.status, stdout: r.stdout ?? '', stderr: `${r.stderr ?? ''}${why}` };
|
||||
}
|
||||
|
||||
function cap(text, max) {
|
||||
|
|
|
|||
|
|
@ -23,8 +23,8 @@ import { appendFileSync, existsSync, readFileSync, realpathSync } from 'node:fs'
|
|||
import { spawnSync } from 'node:child_process';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { allowedCommand, screenCommand } from './criteria-runner.mjs';
|
||||
import { commandEnv, DEFAULT_TIMEOUT_MS, repoRoot } from './red-proof.mjs';
|
||||
import { allowedCommand, runCommand, screenCommand } from './criteria-runner.mjs';
|
||||
import { DEFAULT_TIMEOUT_MS, repoRoot } from './red-proof.mjs';
|
||||
|
||||
export function doneLineRe(step, commit) {
|
||||
const sha = commit ? String(commit).replace(/[^0-9a-f]/g, '') : '[0-9a-f]{40}';
|
||||
|
|
@ -55,7 +55,8 @@ export function writeDoneLine({ cwd, step, cmd, report, timeoutMs = DEFAULT_TIME
|
|||
if (!screen.allowed) return { status: 'not-run', reason: `command blocked: ${screen.rule}` };
|
||||
const head = spawnSync('git', ['rev-parse', '--verify', '--quiet', 'HEAD'], { cwd: dir, encoding: 'utf8' });
|
||||
if (head.status !== 0) return { status: 'not-run', reason: `no git HEAD in ${dir}` };
|
||||
const r = spawnSync('/bin/sh', ['-c', cmd], { cwd: dir, env: commandEnv(), encoding: 'utf8', timeout: timeoutMs, maxBuffer: 64 * 1024 * 1024 });
|
||||
// No shell: the words the screen checked are the words that run.
|
||||
const r = runCommand(cmd, { cwd: dir, timeoutMs });
|
||||
if (r.error || r.status === null || r.status === 126 || r.status === 127) {
|
||||
return { status: 'not-run', reason: `the command did not run to an exit code (${r.error?.message ?? r.signal ?? `exit ${r.status}`})`, exit: r.status };
|
||||
}
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@ import { readFileSync, existsSync, realpathSync } from 'node:fs';
|
|||
import { spawnSync } from 'node:child_process';
|
||||
import { resolve, relative, dirname, isAbsolute, sep } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { sliceSteps, parseStepFields, fieldPaths } from '../parsers/plan-schema.mjs';
|
||||
import { sliceSteps, parseStepFields } from '../parsers/plan-schema.mjs';
|
||||
import { parseManifest } from '../parsers/manifest-yaml.mjs';
|
||||
|
||||
// Files Voyage's own process writes while planning and executing.
|
||||
|
|
@ -44,8 +44,9 @@ export const PROCESS_FILES = Object.freeze([
|
|||
]);
|
||||
|
||||
const BACKTICK_SPAN = /`([^`\n]+)`/g;
|
||||
// A backticked path in `Test first:` (spaces allowed: `docs/my file.md`).
|
||||
// Backticked: spaces allowed (`docs/my file.md`); bare: no whitespace.
|
||||
const looksLikeTickedPath = s => /[./]/.test(s) && !/^https?:/.test(s) && !s.startsWith('-') && s.trim() === s;
|
||||
const looksLikePath = s => !/\s/.test(s) && looksLikeTickedPath(s);
|
||||
|
||||
function pathsIn(text) {
|
||||
const out = [];
|
||||
|
|
@ -61,9 +62,20 @@ export function plannedPaths(planText) {
|
|||
for (const s of sliceSteps(planText)) {
|
||||
const f = parseStepFields(s.body);
|
||||
const files = f.get('Files');
|
||||
// One reader with the tests-only check (plan-schema.mjs fieldPaths):
|
||||
// backticked spans as written, else every path-like word.
|
||||
if (files) for (const p of fieldPaths(files.text)) out.add(norm(p));
|
||||
// v6.2.0's own reading, kept as it was (operator 2026-10-11): every
|
||||
// backticked span that looks like a path, and plain words only when the
|
||||
// field has none. A field that names its paths in backticks plans those,
|
||||
// never a path its description mentions. Release review of 6.3.0, rounds
|
||||
// 3-5: reading Files: through the tests-only check's reader
|
||||
// (plan-schema.mjs fieldPaths) dropped paths v6.2.0 planned and planned
|
||||
// directories a description named, so K6 has its own reader again; the
|
||||
// two answer different questions (what may change; is every named file a test).
|
||||
if (files) {
|
||||
const ticked = pathsIn(files.text);
|
||||
// Older plans write bare paths: `- Files: a.ts, b.ts`.
|
||||
const bare = ticked.length ? [] : files.text.split(/[,\s]+/).filter(looksLikePath);
|
||||
for (const p of [...ticked, ...bare]) out.add(norm(p));
|
||||
}
|
||||
const tf = f.get('Test first');
|
||||
if (tf) {
|
||||
for (const line of tf.text.split('\n')) {
|
||||
|
|
|
|||
|
|
@ -42,6 +42,7 @@
|
|||
// (criteria-runner.mjs `allowedCommand`), and it is screened through the
|
||||
// plugin's own PreToolUse denylist (`screenCommand`): a command spawned from
|
||||
// node never passes through the Bash tool, so the hook cannot fire by itself.
|
||||
// It runs with no shell (`runCommand`): the words checked are the words run.
|
||||
//
|
||||
// When the output is TAP (`--test-reporter=tap`, pytest-tap, ...), the counts
|
||||
// `# tests N` / `# fail N` are reported per run. They show WHAT the red was:
|
||||
|
|
@ -58,7 +59,7 @@ import { spawnSync } from 'node:child_process';
|
|||
import { tmpdir } from 'node:os';
|
||||
import { dirname, isAbsolute, join, resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { allowedCommand, screenCommand } from './criteria-runner.mjs';
|
||||
import { allowedCommand, runCommand, screenCommand } from './criteria-runner.mjs';
|
||||
import { commandEnv } from '../util/command-env.mjs';
|
||||
import { sliceSteps, stepTestsOnly } from '../parsers/plan-schema.mjs';
|
||||
|
||||
|
|
@ -90,8 +91,9 @@ export function repoRoot(dir) {
|
|||
// failing test (independent review, 2026-10-08).
|
||||
const LOAD_ERROR = /^\s*SyntaxError: |ERR_MODULE_NOT_FOUND|does not provide an export named|Cannot find module|ERR_REQUIRE_ESM|ModuleNotFoundError|ImportError: /m;
|
||||
|
||||
// No shell: the words the screen checked are the words that run.
|
||||
function run(cmd, cwd, timeoutMs) {
|
||||
const r = spawnSync('/bin/sh', ['-c', cmd], { cwd, env: commandEnv(), encoding: 'utf8', timeout: timeoutMs, maxBuffer: 64 * 1024 * 1024 });
|
||||
const r = runCommand(cmd, { cwd, timeoutMs });
|
||||
const output = `${r.stdout ?? ''}${r.stderr ?? ''}`;
|
||||
return { exit: r.status, signal: r.signal ?? null, tap: tap(output), tail: output.slice(-TAIL), spawnError: r.error ? r.error.message : null, loadError: LOAD_ERROR.test(output) };
|
||||
}
|
||||
|
|
@ -118,12 +120,13 @@ function testsOnly(o) {
|
|||
return { tests_only: stepTestsOnly(step.body).testsOnly };
|
||||
}
|
||||
|
||||
// A shell that could not find or execute the command (126/127), a signal (the
|
||||
// timeout), or a spawn error is not a test verdict at all.
|
||||
// A program that could not start (a spawn error: not installed), a signal (the
|
||||
// timeout), or 126/127 from a script that could not run its own command is not
|
||||
// a test verdict at all.
|
||||
function unrunnable(r) {
|
||||
if (r.spawnError) return `the command could not start: ${r.spawnError}`;
|
||||
if (r.exit === null) return `the command was killed (${r.signal ?? 'timeout'})`;
|
||||
if (r.exit === 126 || r.exit === 127) return `the shell could not run the command (exit ${r.exit})`;
|
||||
if (r.exit === 126 || r.exit === 127) return `the command could not be run (exit ${r.exit})`;
|
||||
return null;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -26,24 +26,17 @@
|
|||
// The runner reports `exitCode`; this CLI writes `exit_code`, the key
|
||||
// progress.json uses (C15). Nothing else is translated.
|
||||
//
|
||||
// The command runs the way the runner runs it (`/bin/sh -c`), but without
|
||||
// NODE_TEST_CONTEXT in its environment (lib/util/command-env.mjs): measured
|
||||
// 2026-10-08, a red `node --test` regression read as passed when this CLI was
|
||||
// itself started under `node --test`, because the nested run reported to the
|
||||
// parent and exited 0.
|
||||
// The command runs exactly as the runner runs a criterion: its own run, with
|
||||
// no shell and without NODE_TEST_CONTEXT (criteria-runner.mjs `runCommand`).
|
||||
// This file had a copy of that code, and a fix made in one copy would have
|
||||
// left K10 regressions running differently from Phase 7 criteria (release
|
||||
// review of 6.3.0, round 5).
|
||||
|
||||
import { readFileSync, realpathSync } from 'node:fs';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { sliceSteps, parseStepFields } from '../parsers/plan-schema.mjs';
|
||||
import { firstCommand, runCriteria } from './criteria-runner.mjs';
|
||||
import { commandEnv } from '../util/command-env.mjs';
|
||||
|
||||
function exec(command, { cwd, timeoutMs }) {
|
||||
const r = spawnSync('/bin/sh', ['-c', command], { cwd, encoding: 'utf8', timeout: timeoutMs, env: commandEnv() });
|
||||
return { status: r.status, stdout: r.stdout ?? '', stderr: r.stderr ?? '' };
|
||||
}
|
||||
|
||||
export const REGRESSION_FIELD = 'Regression';
|
||||
const EXIT = Object.freeze({ passed: 0, absent: 0, failed: 1, blocked: 1, unrunnable: 1, error: 2 });
|
||||
|
|
@ -54,7 +47,7 @@ const withProgress = (r) => ({ ...r, progress_status: PROGRESS[r.status] ?? 'not
|
|||
* @param {{ planText: string, step: number, cwd?: string, run?: Function }} o
|
||||
* @returns {{ step: number, status: string, command: string|null, exit_code: number|null, output: string, exit: 0|1|2 }}
|
||||
*/
|
||||
export function checkRegression({ planText, step, cwd = process.cwd(), run = (c, o) => runCriteria(c, { ...o, exec }) }) {
|
||||
export function checkRegression({ planText, step, cwd = process.cwd(), run = (c, o) => runCriteria(c, o) }) {
|
||||
const base = { step, command: null, exit_code: null, output: '' };
|
||||
const found = sliceSteps(planText).find((s) => Number(s.n) === step);
|
||||
if (!found) return withProgress({ ...base, status: 'error', output: `no "### Step ${step}:" in the plan`, exit: EXIT.error });
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue