fix: round 8 of the release review of 6.3.0, for 6.3.1
Round 8 (0f400d3..29b4ac9, BLOCK: 1 BLOCKER, 12 MAJOR, 1 MINOR). Each fix
began with a test that failed on 29b4ac9 (13 tests, all red there); suite
2221 (2219/0/2).
- BLOCKER: a backticked check whose trailing comment holds a quote, $,
backtick or backslash is refused, not dropped. The comment was never
screened, while the executor hands the Verify span as written to red
proof, the test lock and the done line inside --cmd "...", where a $(...)
runs and a quote ends the argument. trekplan's text and the reviewer's
NOT RUN table say so.
- A span that is only a comment is not-a-command, not a placeholder. The
dropComment text says the allowlist still refuses a word with a # in it.
- The criteria runner, comparison-gate and end-state-gate use isMain: they
did nothing and exited 0 under a symlinked path. The test now refuses any
process.argv[1] comparison outside lib/util/is-main.mjs.
- Files: in a field with backticked paths, a plain word counts only before
the entry's description (a dash, or a colon after the path); a name with
no slash only first in its entry or after and/or/+/&. "mocks
fs.readFileSync", "runs on Node.js 22.x" or "no change to package.json"
no longer make a tests-only step look like it builds something.
Lowercase makefile, rakefile and justfile are build files. Measured on the
520 Files: fields in 48 plans of the marketplace and the test repo:
3 fields read differently (two notes no longer read as files, one
duplicate name dropped), tests-only changed in 0.
- The test lock compares --cmd without its trailing comment, and the test
lock and the done line read a stop with stoppedBy, the one reading.
red-proof drops its second reading, spawnError.
- release-check reads a relative --review in --repo or, when only that one
exists, from the current directory as 6.3.0 did; two different files are
exit 2, never guessed. The runbook says so.
- One trailing-json reader: a closing fence may be longer than three
backticks, in the block reader and the verdict gate alike, and the review
validator and the synthesis digest use it, so three backticks inside a
JSON string no longer hide a BLOCKER from the remediation-step check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
29b4ac91b3
commit
a10018f9d3
18 changed files with 310 additions and 56 deletions
|
|
@ -111,6 +111,7 @@ column names the reason the runner gave:
|
||||||
| `placeholder` | the text is a template placeholder (`{exact command}`) |
|
| `placeholder` | the text is a template placeholder (`{exact command}`) |
|
||||||
| `not-a-command` | the sentence opened by NAMING a flag or a path (`--verbose`, `tests/`) rather than by invoking something. Running it would have produced exit 2 or exit 126 — a number about the sentence, not about the code |
|
| `not-a-command` | the sentence opened by NAMING a flag or a path (`--verbose`, `tests/`) rather than by invoking something. Running it would have produced exit 2 or exit 126 — a number about the sentence, not about the code |
|
||||||
| `outside the allowlist` | the command is not a known test-runner invocation, or it carries a shell operator. It was never run — say so; do not treat it as a broken criterion |
|
| `outside the allowlist` | the command is not a known test-runner invocation, or it carries a shell operator. It was never run — say so; do not treat it as a broken criterion |
|
||||||
|
| `a trailing comment with a quote, $, backtick or backslash` | the backticked command ends in a `#` comment that a shell would still read inside double quotes. It was never run — say so; do not treat it as a broken criterion |
|
||||||
| `the program could not start` | the runner is not installed where the review ran. Nothing was measured — say so; do not treat it as a broken criterion |
|
| `the program could not start` | the runner is not installed where the review ran. Nothing was measured — say so; do not treat it as a broken criterion |
|
||||||
|
|
||||||
In every one of those cases, judge the criterion on delivered code alone and
|
In every one of those cases, judge the criterion on delivered code alone and
|
||||||
|
|
|
||||||
|
|
@ -797,7 +797,9 @@ Write the plan following the template structure. The plan MUST include:
|
||||||
`node --test` on Node 21 and later (which expands its own patterns), no
|
`node --test` on Node 21 and later (which expands its own patterns), no
|
||||||
path climbs out with `..` or starts at `/` (git included), each runner
|
path climbs out with `..` or starts at `/` (git included), each runner
|
||||||
takes a short list of read-only options, and nothing may follow
|
takes a short list of read-only options, and nothing may follow
|
||||||
`make test`. A trailing `# comment` is dropped, on a line of a shell fence and in a backticked command alike. Anything else is reported NOT
|
`make test`. A trailing `# comment` is dropped, on a line of a shell fence and in a backticked command alike;
|
||||||
|
in a backticked command it may not hold a quote, `$`, backtick or backslash (the executor passes the
|
||||||
|
span on inside double quotes), and such a comment is refused, not dropped. Anything else is reported NOT
|
||||||
RUN, never executed and never failed. A check that needs more than that is
|
RUN, never executed and never failed. A check that needs more than that is
|
||||||
wrapped in a script under `tests/` and declared as `bash tests/<script>.sh`.
|
wrapped in a script under `tests/` and declared as `bash tests/<script>.sh`.
|
||||||
9. **Estimated Scope** — File counts and complexity rating.
|
9. **Estimated Scope** — File counts and complexity rating.
|
||||||
|
|
|
||||||
|
|
@ -79,7 +79,9 @@ do. `scripts/release-check.mjs` refuses a release whose section has no
|
||||||
```
|
```
|
||||||
|
|
||||||
From another directory, add `--repo <dir>`: a relative `--review` is then
|
From another directory, add `--repo <dir>`: a relative `--review` is then
|
||||||
read in that directory, like every other path the check reads.
|
read in that directory, like every other path the check reads, or from
|
||||||
|
where you stand when only that file exists. When both exist and differ,
|
||||||
|
the check does not guess: exit 2, pass an absolute path.
|
||||||
|
|
||||||
It checks, in order: the working tree is clean (otherwise the suite would
|
It checks, in order: the working tree is clean (otherwise the suite would
|
||||||
not measure HEAD), `npm test` exits 0 at HEAD, the review verdict is not
|
not measure HEAD), `npm test` exits 0 at HEAD, the review verdict is not
|
||||||
|
|
|
||||||
|
|
@ -296,10 +296,15 @@ export function onFailureAction(text) {
|
||||||
* wherever they sit (`a.mjs`/`b.mjs`, inside a note), a backticked build
|
* wherever they sit (`a.mjs`/`b.mjs`, inside a note), a backticked build
|
||||||
* file (`Makefile`, `Dockerfile`), and a backticked capitalised name with no
|
* file (`Makefile`, `Dockerfile`), and a backticked capitalised name with no
|
||||||
* extension that stands alone in its entry. Beside them a plain word counts
|
* extension that stands alone in its entry. Beside them a plain word counts
|
||||||
* only when it is clearly a file or a directory: a name with an extension, a
|
* only when it is clearly a file or a directory, and only before the entry's
|
||||||
* dotfile, a path with a slash that ends in a file or a `/`, or a build file
|
* description starts (a dash, or a colon after the backticked path): a path
|
||||||
* (", Dockerfile", "the CLI entry src/cli.mjs", "and Makefile"; round 7: they
|
* with a slash that ends in a file or a `/` anywhere there ("the CLI entry
|
||||||
* were dropped, and a step that builds something read as tests-only).
|
* src/cli.mjs"), and a name with an extension, a dotfile or a build file only
|
||||||
|
* where an entry or a list starts: first in its entry, or after "and", "or",
|
||||||
|
* "+" or "&" (", Dockerfile", "and Makefile", ", package.json"). Round 7: they
|
||||||
|
* were dropped, and a step that builds something read as tests-only. Round 8:
|
||||||
|
* "— mocks fs.readFileSync", "runs on Node.js 22.x" or "no change to
|
||||||
|
* package.json" read as files, and a step that holds only its test did not.
|
||||||
* "covers pass/fail", "JSON, YAML" or "v6.3.0 format" is no path.
|
* "covers pass/fail", "JSON, YAML" or "v6.3.0 format" is no path.
|
||||||
* - A field with no backticked path names every path-like word: a name with an
|
* - A field with no backticked path names every path-like word: a name with an
|
||||||
* extension, a dotfile, a path with a slash, a directory (`docs/`), a glob
|
* extension, a dotfile, a path with a slash, a directory (`docs/`), a glob
|
||||||
|
|
@ -318,8 +323,12 @@ const BARE_NAME = /^[A-Z][\w.-]*$/;
|
||||||
// The last segment is a dotfile or a name with an extension, or the path is a directory.
|
// The last segment is a dotfile or a name with an extension, or the path is a directory.
|
||||||
const FILE_OR_DIR = /(?:^|\/)(?:\.[\w-][\w.-]*|[^/]*\.[A-Za-z*][A-Za-z0-9*]*)$|\/$/;
|
const FILE_OR_DIR = /(?:^|\/)(?:\.[\w-][\w.-]*|[^/]*\.[A-Za-z*][A-Za-z0-9*]*)$|\/$/;
|
||||||
// A build file named without an extension: a capitalised name that ends in
|
// A build file named without an extension: a capitalised name that ends in
|
||||||
// `file` (Makefile, Dockerfile, Gemfile, Jenkinsfile, ...), not the word Profile.
|
// `file` (Makefile, Dockerfile, Gemfile, Jenkinsfile, ...), not the word Profile,
|
||||||
const BUILD_FILE = /^(?!Profile$)(?:[A-Z][a-z]+|GNUmake)file$/;
|
// and the lowercase names make, rake and just also read (round 8).
|
||||||
|
const BUILD_FILE = /^(?!Profile$)(?:[A-Z][a-z]+|GNUmake)file$|^(?:make|rake|just)file$/;
|
||||||
|
// Where an entry's description starts, and the words that start a list item.
|
||||||
|
const DESCRIPTION_DASH = /^(?:\u2014|\u2013|-|--)$/;
|
||||||
|
const LIST_JOINER = /^(?:and|or|\+|&)$/i;
|
||||||
const lastSegment = (p) => p.replace(/\/$/, '').split('/').pop();
|
const lastSegment = (p) => p.replace(/\/$/, '').split('/').pop();
|
||||||
const norm = (p) => p.replace(/\\/g, '/').replace(/^\.\//, '');
|
const norm = (p) => p.replace(/\\/g, '/').replace(/^\.\//, '');
|
||||||
// A backticked span that is a path, not a URL, an option or a call (`parse()`, `p.parse()`).
|
// A backticked span that is a path, not a URL, an option or a call (`parse()`, `p.parse()`).
|
||||||
|
|
@ -346,11 +355,21 @@ export function fieldPaths(text) {
|
||||||
const alone = /^\u0000(\d+)\u0000$/.exec(entry);
|
const alone = /^\u0000(\d+)\u0000$/.exec(entry);
|
||||||
if (alone && BARE_NAME.test(spans[Number(alone[1])])) { out.push(spans[Number(alone[1])]); continue; }
|
if (alone && BARE_NAME.test(spans[Number(alone[1])])) { out.push(spans[Number(alone[1])]); continue; }
|
||||||
if (tickedField) {
|
if (tickedField) {
|
||||||
// In the order they stand: the backticked paths, and a plain word only
|
// In the order they stand: the backticked paths, and before the
|
||||||
// when it is clearly a file or a directory (round 7).
|
// description a plain word that is clearly a file where it may stand
|
||||||
for (const t of tokens) {
|
// (rounds 7-8).
|
||||||
if (t.includes('\u0000')) out.push(...[...t.matchAll(/\u0000(\d+)\u0000/g)].map((m) => Number(m[1])).filter((i) => isTicked[i]).map((i) => spans[i]));
|
let described = false;
|
||||||
else if (isFileWord(plainWord(t))) out.push(plainWord(t));
|
for (let i = 0; i < tokens.length; i++) {
|
||||||
|
const t = tokens[i];
|
||||||
|
if (t.includes('\u0000')) {
|
||||||
|
out.push(...[...t.matchAll(/\u0000(\d+)\u0000/g)].map((m) => Number(m[1])).filter((n) => isTicked[n]).map((n) => spans[n]));
|
||||||
|
if (t.endsWith(':')) described = true;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (DESCRIPTION_DASH.test(t)) described = true;
|
||||||
|
const w = plainWord(t);
|
||||||
|
if (described || !isFileWord(w)) continue;
|
||||||
|
if (w.includes('/') || i === 0 || LIST_JOINER.test(tokens[i - 1])) out.push(w);
|
||||||
}
|
}
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -30,6 +30,7 @@
|
||||||
import { readFileSync, existsSync } from 'node:fs';
|
import { readFileSync, existsSync } from 'node:fs';
|
||||||
import { issue, fail } from '../util/result.mjs';
|
import { issue, fail } from '../util/result.mjs';
|
||||||
import { isMain } from '../util/is-main.mjs';
|
import { isMain } from '../util/is-main.mjs';
|
||||||
|
import { extractFindingsBlock } from '../review/findings-schema.mjs';
|
||||||
|
|
||||||
// Origin tag for every synthesised finding (Phase 7 rule 7: codebase vs research).
|
// Origin tag for every synthesised finding (Phase 7 rule 7: codebase vs research).
|
||||||
export const ORIGIN_VALUES = Object.freeze(['codebase', 'research']);
|
export const ORIGIN_VALUES = Object.freeze(['codebase', 'research']);
|
||||||
|
|
@ -55,23 +56,15 @@ const ARRAY_FIELDS = Object.freeze([
|
||||||
['sources', 'DIGEST_SOURCES_NOT_ARRAY'],
|
['sources', 'DIGEST_SOURCES_NOT_ARRAY'],
|
||||||
]);
|
]);
|
||||||
|
|
||||||
// Last fenced ```json … ``` block, so prose above it never confuses the parser.
|
|
||||||
const JSON_FENCE_GLOBAL = /```json[ \t]*\r?\n([\s\S]*?)```/gi;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Extract the inner body of the LAST fenced `json` block in `text`.
|
* Extract the inner body of the LAST fenced `json` block in `text`, with the
|
||||||
|
* one block reader (lib/review/findings-schema.mjs): three backticks inside a
|
||||||
|
* JSON string do not end the block (release review of 6.3.0, round 8).
|
||||||
* @param {string} text
|
* @param {string} text
|
||||||
* @returns {string|null} the JSON source, or null if no json fence is present.
|
* @returns {string|null} the JSON source, or null if no json fence is present.
|
||||||
*/
|
*/
|
||||||
export function extractDigestBlock(text) {
|
export function extractDigestBlock(text) {
|
||||||
if (typeof text !== 'string') return null;
|
return extractFindingsBlock(text);
|
||||||
JSON_FENCE_GLOBAL.lastIndex = 0;
|
|
||||||
let last = null;
|
|
||||||
let m;
|
|
||||||
while ((m = JSON_FENCE_GLOBAL.exec(text)) !== null) {
|
|
||||||
last = m[1];
|
|
||||||
}
|
|
||||||
return last;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
function isNonEmptyString(v) {
|
function isNonEmptyString(v) {
|
||||||
|
|
|
||||||
|
|
@ -42,8 +42,12 @@ export const FINDING_REQUIRED_FIELDS = Object.freeze([
|
||||||
// gate's reader: a JSON string cannot hold a line break, so three backticks
|
// gate's reader: a JSON string cannot hold a line break, so three backticks
|
||||||
// inside one ("a fenced ```bash block") never end the block. Ending it at the
|
// inside one ("a fenced ```bash block") never end the block. Ending it at the
|
||||||
// first triple backtick anywhere made fill-finding-ids exit 2 (release review
|
// first triple backtick anywhere made fill-finding-ids exit 2 (release review
|
||||||
// of 6.3.0, found while writing round 7's review.md; present in 6.2.0).
|
// of 6.3.0, found while writing round 7's review.md; present in 6.2.0). As in
|
||||||
const JSON_FENCE_GLOBAL = /^[ \t]*```json[ \t]*\r?\n([\s\S]*?)^[ \t]*```[ \t]*\r?$/gim;
|
// Markdown, the closing fence may be longer than three backticks (round 8);
|
||||||
|
// three backticks after text on the same line are no fence, here or in the gate.
|
||||||
|
// One reader for every trailing json block: the review validator and the
|
||||||
|
// synthesis digest import it.
|
||||||
|
const JSON_FENCE_GLOBAL = /^[ \t]*```json[ \t]*\r?\n([\s\S]*?)^[ \t]*`{3,}[ \t]*\r?$/gim;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Extract the inner body of the LAST fenced `json` block in `text`.
|
* Extract the inner body of the LAST fenced `json` block in `text`.
|
||||||
|
|
|
||||||
|
|
@ -118,7 +118,9 @@ function atLeastVersion(v, major, minor) {
|
||||||
return Boolean(m) && (Number(m[1]) > major || (Number(m[1]) === major && Number(m[2]) >= minor));
|
return Boolean(m) && (Number(m[1]) > major || (Number(m[1]) === major && Number(m[2]) >= minor));
|
||||||
}
|
}
|
||||||
|
|
||||||
const FENCE_RE = /^```[ \t]*([A-Za-z0-9_+-]*)[^\n]*\n[\s\S]*?^```[ \t]*$/gm;
|
// A fence as lib/review/findings-schema.mjs reads one: on a line of its own,
|
||||||
|
// indented or not, and a closing fence of three backticks or more (round 8).
|
||||||
|
const FENCE_RE = /^[ \t]*```[ \t]*([A-Za-z0-9_+-]*)[^\n]*\n[\s\S]*?^[ \t]*`{3,}[ \t]*$/gm;
|
||||||
|
|
||||||
/** Language tag of the LAST fenced block in `body` (lower-cased), '' for none, null when no fence. */
|
/** Language tag of the LAST fenced block in `body` (lower-cased), '' for none, null when no fence. */
|
||||||
function lastFenceLanguage(body) {
|
function lastFenceLanguage(body) {
|
||||||
|
|
|
||||||
|
|
@ -42,7 +42,7 @@ import { join, resolve, dirname } from 'node:path';
|
||||||
import { createHash } from 'node:crypto';
|
import { createHash } from 'node:crypto';
|
||||||
import { tmpdir } from 'node:os';
|
import { tmpdir } from 'node:os';
|
||||||
import { resolveRepo, canonicalPath } from '../verification/repo-tools.mjs';
|
import { resolveRepo, canonicalPath } from '../verification/repo-tools.mjs';
|
||||||
import { DEFAULT_TIMEOUT_MS, allowedCommand, runCommand, screenCommand } from '../verification/criteria-runner.mjs';
|
import { DEFAULT_TIMEOUT_MS, allowedCommand, dropComment, runCommand, screenCommand, stoppedBy } from '../verification/criteria-runner.mjs';
|
||||||
import { isMain } from './is-main.mjs';
|
import { isMain } from './is-main.mjs';
|
||||||
|
|
||||||
export const LOCK_VERSION = 1;
|
export const LOCK_VERSION = 1;
|
||||||
|
|
@ -172,7 +172,8 @@ function main(argv) {
|
||||||
} else {
|
} else {
|
||||||
const testCmd = typeof lock.cmd === 'string' && lock.cmd.trim() !== '' ? lock.cmd : null;
|
const testCmd = typeof lock.cmd === 'string' && lock.cmd.trim() !== '' ? lock.cmd : null;
|
||||||
if (!testCmd) fail(2, 'the lock names no test command (lock with --cmd "<Verify>"); never marked green');
|
if (!testCmd) fail(2, 'the lock names no test command (lock with --cmd "<Verify>"); never marked green');
|
||||||
if (args.cmd !== undefined && args.cmd !== testCmd) fail(2, `--cmd is not the step's test command stored in the lock (${testCmd})`);
|
// One reading of a command passed as written: a trailing comment is no part of it (round 8).
|
||||||
|
if (args.cmd !== undefined && dropComment(args.cmd) !== dropComment(testCmd)) fail(2, `--cmd is not the step's test command stored in the lock (${testCmd})`);
|
||||||
const allowed = allowedCommand(testCmd, { cwd: cur.repo.root });
|
const allowed = allowedCommand(testCmd, { cwd: cur.repo.root });
|
||||||
if (!allowed.allowed) fail(2, `command not run: ${allowed.reason}`);
|
if (!allowed.allowed) fail(2, `command not run: ${allowed.reason}`);
|
||||||
const screen = screenCommand(testCmd);
|
const screen = screenCommand(testCmd);
|
||||||
|
|
@ -180,9 +181,9 @@ function main(argv) {
|
||||||
// No shell: the words the screen checked are the words that run.
|
// No shell: the words the screen checked are the words that run.
|
||||||
const r = runCommand(testCmd, { cwd: cur.repo.root, timeoutMs: DEFAULT_TIMEOUT_MS });
|
const r = runCommand(testCmd, { cwd: cur.repo.root, timeoutMs: DEFAULT_TIMEOUT_MS });
|
||||||
// With no shell, 126/127 is the test program's own exit: a red, not "could not run".
|
// With no shell, 126/127 is the test program's own exit: a red, not "could not run".
|
||||||
if (r.error || r.status === null) {
|
// A stop is read once, by stoppedBy (round 8).
|
||||||
fail(2, `the test command did not run to an exit code (${r.error?.message ?? r.signal ?? `exit ${r.status}`})`);
|
const stopped = stoppedBy(r, { timeoutMs: DEFAULT_TIMEOUT_MS });
|
||||||
}
|
if (stopped) fail(2, `the test command ${stopped.why}`);
|
||||||
if (r.status !== 0) fail(1, `the test command exited ${r.status}; the lock stays unmarked`);
|
if (r.status !== 0) fail(1, `the test command exited ${r.status}; the lock stays unmarked`);
|
||||||
lock.green_at = new Date().toISOString();
|
lock.green_at = new Date().toISOString();
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -17,6 +17,7 @@ import { readFileSync, existsSync } from 'node:fs';
|
||||||
import { parseDocument } from '../util/frontmatter.mjs';
|
import { parseDocument } from '../util/frontmatter.mjs';
|
||||||
import { issue, ok, fail } from '../util/result.mjs';
|
import { issue, ok, fail } from '../util/result.mjs';
|
||||||
import { isMain } from '../util/is-main.mjs';
|
import { isMain } from '../util/is-main.mjs';
|
||||||
|
import { extractFindingsBlock } from '../review/findings-schema.mjs';
|
||||||
|
|
||||||
export const REVIEW_REQUIRED_FRONTMATTER = [
|
export const REVIEW_REQUIRED_FRONTMATTER = [
|
||||||
'type',
|
'type',
|
||||||
|
|
@ -33,14 +34,13 @@ export const REVIEW_BODY_SECTIONS = ['Executive Summary', 'Coverage', 'Remediati
|
||||||
export const VERDICT_VALUES = Object.freeze(['ALLOW', 'WARN', 'BLOCK']);
|
export const VERDICT_VALUES = Object.freeze(['ALLOW', 'WARN', 'BLOCK']);
|
||||||
|
|
||||||
const HEX_ID_RE = /^[0-9a-f]{40}$/;
|
const HEX_ID_RE = /^[0-9a-f]{40}$/;
|
||||||
const JSON_FENCE_GLOBAL = /```json[ \t]*\r?\n([\s\S]*?)```/gi;
|
/**
|
||||||
|
* IDs of BLOCKER findings in the trailing JSON block ([] when absent/unparseable).
|
||||||
/** IDs of BLOCKER findings in the trailing JSON block ([] when absent/unparseable). */
|
* Read with the one block reader, so three backticks inside a JSON string do
|
||||||
|
* not end the block early and hide a BLOCKER (release review of 6.3.0, round 8).
|
||||||
|
*/
|
||||||
function blockerIdsFromBody(body) {
|
function blockerIdsFromBody(body) {
|
||||||
let last = null;
|
const last = extractFindingsBlock(body);
|
||||||
let m;
|
|
||||||
JSON_FENCE_GLOBAL.lastIndex = 0;
|
|
||||||
while ((m = JSON_FENCE_GLOBAL.exec(body)) !== null) last = m[1];
|
|
||||||
if (last === null) return [];
|
if (last === null) return [];
|
||||||
try {
|
try {
|
||||||
const payload = JSON.parse(last);
|
const payload = JSON.parse(last);
|
||||||
|
|
|
||||||
|
|
@ -50,6 +50,7 @@ import { fileURLToPath } from 'node:url';
|
||||||
import { spawnSync } from 'node:child_process';
|
import { spawnSync } from 'node:child_process';
|
||||||
import { sliceSteps, parseStepFields } from '../parsers/plan-schema.mjs';
|
import { sliceSteps, parseStepFields } from '../parsers/plan-schema.mjs';
|
||||||
import { commandEnv } from '../util/command-env.mjs';
|
import { commandEnv } from '../util/command-env.mjs';
|
||||||
|
import { isMain } from '../util/is-main.mjs';
|
||||||
|
|
||||||
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
||||||
|
|
||||||
|
|
@ -149,16 +150,26 @@ export function looksLikeCommand(span) {
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const COMMENT_NOT_PLAIN = 'a trailing comment with a quote, $, backtick or backslash (the span reaches a shell inside double quotes)';
|
||||||
|
|
||||||
// The first backtick-delimited span on the line is the command by convention —
|
// The first backtick-delimited span on the line is the command by convention —
|
||||||
// both templates put it first and a second span holds the expected output.
|
// both templates put it first and a second span holds the expected output.
|
||||||
// A plan step's `Verify:` and `End-to-end:` fields follow the same convention.
|
// A plan step's `Verify:` and `End-to-end:` fields follow the same convention.
|
||||||
export function firstCommand(text) {
|
export function firstCommand(text) {
|
||||||
const m = text.match(/`([^`]+)`/);
|
const m = text.match(/`([^`]+)`/);
|
||||||
if (!m) return { command: null, reason: 'no-command' };
|
if (!m) return { command: null, reason: 'no-command' };
|
||||||
const raw = dropComment(m[1]);
|
const span = m[1].trim();
|
||||||
|
const raw = dropComment(span);
|
||||||
// Template placeholders (`{exact command}`) are not commands.
|
// Template placeholders (`{exact command}`) are not commands.
|
||||||
if (raw === '' || /^\{.*\}$/.test(raw)) return { command: null, reason: 'placeholder' };
|
if (span === '' || /^\{.*\}$/.test(raw)) return { command: null, reason: 'placeholder' };
|
||||||
if (!looksLikeCommand(raw)) return { command: null, reason: 'not-a-command' };
|
// A span that is only a comment names nothing to run (round 8).
|
||||||
|
if (raw === '' || !looksLikeCommand(raw)) return { command: null, reason: 'not-a-command' };
|
||||||
|
// The comment is never screened, but the executor hands the span as written
|
||||||
|
// to red proof, the test lock and the done line inside double quotes
|
||||||
|
// (`--cmd "{Verify command}"`): there a `"` ends the argument and a `$`, a
|
||||||
|
// backtick or a backslash still means something (release review of 6.3.0,
|
||||||
|
// round 8). A comment with one of them is refused, never dropped.
|
||||||
|
if (/["$`\\]/.test(span.slice(raw.length))) return { command: null, reason: COMMENT_NOT_PLAIN };
|
||||||
return { command: raw, reason: '' };
|
return { command: raw, reason: '' };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -170,7 +181,9 @@ export function firstCommand(text) {
|
||||||
* span (`Verify:`, `Regression:`, `End-to-end:`, a `## Verification` bullet)
|
* span (`Verify:`, `Regression:`, `End-to-end:`, a `## Verification` bullet)
|
||||||
* and a command passed as written (red proof, the done line, the test lock),
|
* and a command passed as written (red proof, the done line, the test lock),
|
||||||
* so the words screened are the words run (round 7). Only a `#` that opens a
|
* so the words screened are the words run (round 7). Only a `#` that opens a
|
||||||
* word on the same line is a comment (`tests/a#b.test.mjs` is a path).
|
* word on the same line is a comment: `tests/a#b.test.mjs` is kept whole here,
|
||||||
|
* and the allowlist then refuses the word, since a `#` is not a plain word
|
||||||
|
* (round 8).
|
||||||
*/
|
*/
|
||||||
export function dropComment(command) {
|
export function dropComment(command) {
|
||||||
return String(command ?? '').replace(/[ \t]+#[^\n]*$/, '').trim();
|
return String(command ?? '').replace(/[ \t]+#[^\n]*$/, '').trim();
|
||||||
|
|
@ -887,6 +900,6 @@ export function main(argv) {
|
||||||
return rep.summary.ok ? 0 : 1;
|
return rep.summary.ok ? 0 : 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
if (isMain(import.meta.url)) {
|
||||||
process.exitCode = main(process.argv.slice(2));
|
process.exitCode = main(process.argv.slice(2));
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -23,7 +23,7 @@
|
||||||
import { appendFileSync, existsSync, readFileSync } from 'node:fs';
|
import { appendFileSync, existsSync, readFileSync } from 'node:fs';
|
||||||
import { spawnSync } from 'node:child_process';
|
import { spawnSync } from 'node:child_process';
|
||||||
import { resolve } from 'node:path';
|
import { resolve } from 'node:path';
|
||||||
import { DEFAULT_TIMEOUT_MS, allowedCommand, runCommand, screenCommand } from './criteria-runner.mjs';
|
import { DEFAULT_TIMEOUT_MS, allowedCommand, runCommand, screenCommand, stoppedBy } from './criteria-runner.mjs';
|
||||||
import { repoRoot } from './red-proof.mjs';
|
import { repoRoot } from './red-proof.mjs';
|
||||||
import { isMain } from '../util/is-main.mjs';
|
import { isMain } from '../util/is-main.mjs';
|
||||||
|
|
||||||
|
|
@ -58,9 +58,9 @@ export function writeDoneLine({ cwd, step, cmd, report, timeoutMs = DEFAULT_TIME
|
||||||
if (head.status !== 0) return { status: 'not-run', reason: `no git HEAD in ${dir}` };
|
if (head.status !== 0) return { status: 'not-run', reason: `no git HEAD in ${dir}` };
|
||||||
// No shell: the words the screen checked are the words that run.
|
// No shell: the words the screen checked are the words that run.
|
||||||
const r = runCommand(cmd, { cwd: dir, timeoutMs });
|
const r = runCommand(cmd, { cwd: dir, timeoutMs });
|
||||||
if (r.error || r.status === null) {
|
// A stop is read once, by stoppedBy (round 8).
|
||||||
return { status: 'not-run', reason: `the command did not run to an exit code (${r.error?.message ?? r.signal ?? `exit ${r.status}`})`, exit: r.status };
|
const stopped = stoppedBy(r, { timeoutMs });
|
||||||
}
|
if (stopped) return { status: 'not-run', reason: `the command ${stopped.why}`, exit: r.status };
|
||||||
if (r.status !== 0) return { status: 'failed', reason: `the test command exited ${r.status}`, exit: r.status };
|
if (r.status !== 0) return { status: 'failed', reason: `the test command exited ${r.status}`, exit: r.status };
|
||||||
const line = `DONE step ${step}: \`${cmd}\` exit 0 at ${head.stdout.trim()} ${new Date().toISOString()}`;
|
const line = `DONE step ${step}: \`${cmd}\` exit 0 at ${head.stdout.trim()} ${new Date().toISOString()}`;
|
||||||
appendFileSync(report, `${line}\n`);
|
appendFileSync(report, `${line}\n`);
|
||||||
|
|
|
||||||
|
|
@ -98,7 +98,7 @@ function run(cmd, cwd, timeoutMs) {
|
||||||
const r = runCommand(cmd, { cwd, timeoutMs });
|
const r = runCommand(cmd, { cwd, timeoutMs });
|
||||||
const output = `${r.stdout ?? ''}${r.stderr ?? ''}`;
|
const output = `${r.stdout ?? ''}${r.stderr ?? ''}`;
|
||||||
const stopped = stoppedBy(r, { timeoutMs });
|
const stopped = stoppedBy(r, { timeoutMs });
|
||||||
return { exit: r.status, signal: r.signal ?? null, tap: tap(output), tail: output.slice(-TAIL), spawnError: r.error ? r.error.message : null, stopped: stopped?.why ?? null, loadError: LOAD_ERROR.test(output) };
|
return { exit: r.status, signal: r.signal ?? null, tap: tap(output), tail: output.slice(-TAIL), stopped: stopped?.why ?? null, loadError: LOAD_ERROR.test(output) };
|
||||||
}
|
}
|
||||||
|
|
||||||
/** What the red at the start point was: the test file failing to load, or a test failing. */
|
/** What the red at the start point was: the test file failing to load, or a test failing. */
|
||||||
|
|
|
||||||
|
|
@ -30,6 +30,7 @@ import { homedir } from 'node:os';
|
||||||
import { join, resolve, dirname, relative, isAbsolute, sep } from 'node:path';
|
import { join, resolve, dirname, relative, isAbsolute, sep } from 'node:path';
|
||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
import { execFileSync } from 'node:child_process';
|
import { execFileSync } from 'node:child_process';
|
||||||
|
import { isMain } from '../lib/util/is-main.mjs';
|
||||||
|
|
||||||
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
|
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
|
||||||
|
|
||||||
|
|
@ -336,6 +337,6 @@ export function main(argv) {
|
||||||
return result.green ? 0 : 1;
|
return result.green ? 0 : 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
if (isMain(import.meta.url)) {
|
||||||
process.exitCode = main(process.argv.slice(2));
|
process.exitCode = main(process.argv.slice(2));
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -28,6 +28,7 @@ import { join, resolve, dirname, relative, sep, basename } from 'node:path';
|
||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
import { createHash } from 'node:crypto';
|
import { createHash } from 'node:crypto';
|
||||||
import { spawnSync } from 'node:child_process';
|
import { spawnSync } from 'node:child_process';
|
||||||
|
import { isMain } from '../lib/util/is-main.mjs';
|
||||||
|
|
||||||
export const REGISTRY_PATH = 'scripts/end-state-registry.json';
|
export const REGISTRY_PATH = 'scripts/end-state-registry.json';
|
||||||
export const FROZEN_PATH = 'tests/fixtures/end-state-frozen.json';
|
export const FROZEN_PATH = 'tests/fixtures/end-state-frozen.json';
|
||||||
|
|
@ -579,6 +580,6 @@ export function main(argv) {
|
||||||
return result.green ? 0 : 1;
|
return result.green ? 0 : 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
if (isMain(import.meta.url)) {
|
||||||
process.exitCode = main(process.argv.slice(2));
|
process.exitCode = main(process.argv.slice(2));
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,8 @@
|
||||||
// code, never a summary:
|
// code, never a summary:
|
||||||
//
|
//
|
||||||
// node scripts/release-check.mjs --review <review.md> [--repo <dir>] [--version X.Y.Z] [--json]
|
// node scripts/release-check.mjs --review <review.md> [--repo <dir>] [--version X.Y.Z] [--json]
|
||||||
// (a relative --review is read in --repo, as every other path)
|
// (a relative --review is read in --repo, as every other path, or from the
|
||||||
|
// current directory when only that one exists; two different files: exit 2)
|
||||||
// 0 = ready
|
// 0 = ready
|
||||||
// 1 = not ready — every failed check is listed
|
// 1 = not ready — every failed check is listed
|
||||||
// 2 = could not run: not a git repo, no version to release, the suite could
|
// 2 = could not run: not a git repo, no version to release, the suite could
|
||||||
|
|
@ -36,7 +37,7 @@
|
||||||
|
|
||||||
import { existsSync, readFileSync } from 'node:fs';
|
import { existsSync, readFileSync } from 'node:fs';
|
||||||
import { spawnSync } from 'node:child_process';
|
import { spawnSync } from 'node:child_process';
|
||||||
import { dirname, join, relative, resolve } from 'node:path';
|
import { dirname, isAbsolute, join, relative, resolve } from 'node:path';
|
||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
import { decideVerdict } from '../lib/review/verdict-gate.mjs';
|
import { decideVerdict } from '../lib/review/verdict-gate.mjs';
|
||||||
import { commandEnv } from '../lib/util/command-env.mjs';
|
import { commandEnv } from '../lib/util/command-env.mjs';
|
||||||
|
|
@ -179,6 +180,23 @@ function checkSuite(repo, treeClean, timeoutMs = DEFAULT_TIMEOUT_MS, maxBuffer =
|
||||||
return { id: 'suite', status: 'pass', detail: `npm test exited 0 at HEAD${tail}` };
|
return { id: 'suite', status: 'pass', detail: `npm test exited 0 at HEAD${tail}` };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Where a relative --review is: in --repo, like every other path here (round 7),
|
||||||
|
* or from the caller's directory, as 6.3.0 read it (round 8: the round-7 change
|
||||||
|
* broke a command that worked). The one that exists is taken; two different
|
||||||
|
* files are never guessed between.
|
||||||
|
* @returns {{path: string|null, error?: string}}
|
||||||
|
*/
|
||||||
|
function reviewPath(root, review) {
|
||||||
|
if (!review) return { path: null };
|
||||||
|
if (isAbsolute(review)) return { path: review };
|
||||||
|
const inRepo = resolve(root, review);
|
||||||
|
const fromHere = resolve(review);
|
||||||
|
if (inRepo === fromHere || !existsSync(fromHere)) return { path: inRepo };
|
||||||
|
if (!existsSync(inRepo)) return { path: fromHere };
|
||||||
|
return { path: null, error: `--review ${review} names two files: ${inRepo} (in --repo) and ${fromHere} (from here); pass an absolute path` };
|
||||||
|
}
|
||||||
|
|
||||||
function checkReview(reviewPath) {
|
function checkReview(reviewPath) {
|
||||||
if (!reviewPath) return { check: { id: 'review', status: 'error', detail: 'no --review given — the release has no review verdict to read (K7)' }, text: null };
|
if (!reviewPath) return { check: { id: 'review', status: 'error', detail: 'no --review given — the release has no review verdict to read (K7)' }, text: null };
|
||||||
if (!existsSync(reviewPath)) return { check: { id: 'review', status: 'error', detail: `${reviewPath} does not exist — no review, no release` }, text: null };
|
if (!existsSync(reviewPath)) return { check: { id: 'review', status: 'error', detail: `${reviewPath} does not exist — no review, no release` }, text: null };
|
||||||
|
|
@ -286,8 +304,10 @@ export function releaseCheck({ repo, review, version: explicitVersion, suiteTime
|
||||||
// The cheap checks first: when one of them cannot be measured the answer is
|
// The cheap checks first: when one of them cannot be measured the answer is
|
||||||
// already exit 2, and the suite (minutes) is not run for nothing.
|
// already exit 2, and the suite (minutes) is not run for nothing.
|
||||||
const tree = checkTree(root);
|
const tree = checkTree(root);
|
||||||
// A relative --review is read in --repo, like every other path here (round 7).
|
const where = reviewPath(root, review);
|
||||||
const { check: rev, text } = checkReview(review ? resolve(root, review) : null);
|
const { check: rev, text } = where.error
|
||||||
|
? { check: { id: 'review', status: 'error', detail: where.error }, text: null }
|
||||||
|
: checkReview(where.path);
|
||||||
const coverage = checkCoverage(root, text);
|
const coverage = checkCoverage(root, text);
|
||||||
const door = checkDoor(root, version);
|
const door = checkDoor(root, version);
|
||||||
const undecided = [tree, rev, coverage, door].filter((c) => c.status === 'error');
|
const undecided = [tree, rev, coverage, door].filter((c) => c.status === 'error');
|
||||||
|
|
|
||||||
|
|
@ -107,3 +107,25 @@ test('green: exit 127 from the test program itself is a red -> exit 1, no mark (
|
||||||
assert.match(g.stderr, /exited 127/);
|
assert.match(g.stderr, /exited 127/);
|
||||||
assert.equal(JSON.parse(cli(repo, ['status']).stdout).green_at, null);
|
assert.equal(JSON.parse(cli(repo, ['status']).stdout).green_at, null);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('green (release review of 6.3.0, round 8): a trailing # comment on either side is no part of the command', () => {
|
||||||
|
for (const [lockCmd, greenCmd] of [[`${CMD} # the step's test`, CMD], [CMD, `${CMD} # the step's test`]]) {
|
||||||
|
const repo = makeRepo('main');
|
||||||
|
mkdirSync(join(repo, 'tests'), { recursive: true });
|
||||||
|
writeFileSync(join(repo, 'tests', 'step.test.mjs'), "import { test } from 'node:test';\ntest('step', () => {});\n");
|
||||||
|
assert.equal(cli(repo, ['lock', '--files', 'tests/step.test.mjs', '--step', '3', '--cmd', lockCmd]).code, 0);
|
||||||
|
const g = cli(repo, ['green', '--cmd', greenCmd]);
|
||||||
|
assert.equal(g.code, 0, `lock "${lockCmd}", green "${greenCmd}": ${g.stderr}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('green (release review of 6.3.0, round 8): a test program that is not installed is reported as could not start', () => {
|
||||||
|
const repo = makeRepo('main');
|
||||||
|
mkdirSync(join(repo, 'tests'), { recursive: true });
|
||||||
|
writeFileSync(join(repo, 'tests', 'step.test.mjs'), 'x\n');
|
||||||
|
// `jest` is on the allowlist and not installed where this suite runs.
|
||||||
|
assert.equal(cli(repo, ['lock', '--files', 'tests/step.test.mjs', '--step', '3', '--cmd', 'jest tests/step.test.mjs']).code, 0);
|
||||||
|
const g = cli(repo, ['green', '--cmd', 'jest tests/step.test.mjs']);
|
||||||
|
assert.equal(g.code, 2, g.stderr);
|
||||||
|
assert.match(g.stderr, /could not start/, g.stderr);
|
||||||
|
});
|
||||||
|
|
|
||||||
|
|
@ -874,3 +874,153 @@ test('review 630 r7 / brief note 3: the prose says a git revision may start with
|
||||||
assert.match(read(rel), /`@~1`/, `${rel} names the git revision as the exception`);
|
assert.match(read(rel), /`@~1`/, `${rel} names the git revision as the exception`);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// ==== round 8 of the release review of 6.3.0 (0f400d3..29b4ac9, 2026-10-11), fixed in 6.3.1 ====
|
||||||
|
|
||||||
|
test('review 630 r8 / BLOCKER criteria-runner.mjs:158: a Verify comment that a shell would read inside double quotes is refused', () => {
|
||||||
|
// The executor and the helper hand the Verify span to red proof, the test lock
|
||||||
|
// and the done line as `--cmd "{Verify command}"`: inside double quotes a `$`,
|
||||||
|
// a backtick or a backslash still means something, and a `"` ends the argument.
|
||||||
|
const verify = (span) => buildPlan().replace('`node --test tests/mod1.test.mjs` → expected: pass', () => `\`${span}\` → expected: pass`);
|
||||||
|
const codes = (span) => {
|
||||||
|
const r = validatePlanContent(verify(span), { strict: false });
|
||||||
|
return [...r.errors, ...r.warnings].filter((i) => i.code === 'PLAN_STEP_VERIFY_NOT_RUNNABLE').map((i) => i.message);
|
||||||
|
};
|
||||||
|
for (const span of ['node --test tests/mod1.test.mjs # $(touch x)', 'node --test tests/mod1.test.mjs # expect "fail 1"',
|
||||||
|
'node --test tests/mod1.test.mjs # a back\\slash', 'node --test tests/mod1.test.mjs # ${HOME}']) {
|
||||||
|
assert.equal(codes(span).length, 1, `${span} is refused: ${JSON.stringify(codes(span))}`);
|
||||||
|
}
|
||||||
|
// The control: a plain comment is still dropped, and the check stays runnable.
|
||||||
|
assert.deepEqual(codes('node --test tests/mod1.test.mjs # all green'), []);
|
||||||
|
assert.deepEqual(codes("node --test tests/mod1.test.mjs # it's green"), [], 'a single quote means nothing inside double quotes');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('review 630 r8 / criteria-runner.mjs:158: a span that is only a comment is not-a-command, not a placeholder', async () => {
|
||||||
|
const { firstCommand } = await import('../../lib/verification/criteria-runner.mjs');
|
||||||
|
assert.equal(firstCommand('` # fill in`').reason, 'not-a-command');
|
||||||
|
assert.equal(firstCommand('`{exact command}`').reason, 'placeholder', 'the control');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('review 630 r8 / criteria-runner.mjs:175: a # inside a word is kept by dropComment and refused by the allowlist, and the text says so', async () => {
|
||||||
|
const { firstCommand, allowedCommand } = await import('../../lib/verification/criteria-runner.mjs');
|
||||||
|
assert.equal(firstCommand('`node --test tests/a#b.test.mjs`').command, 'node --test tests/a#b.test.mjs');
|
||||||
|
assert.equal(allowedCommand('node --test tests/a#b.test.mjs', { cwd: ROOT }).allowed, false);
|
||||||
|
const doc = /\/\*\*\n((?: \*[^\n]*\n)+) \*\/\nexport function dropComment/.exec(read('lib/verification/criteria-runner.mjs'));
|
||||||
|
assert.ok(doc, 'dropComment has its doc comment');
|
||||||
|
assert.match(doc[1].replace(/\n \* ?/g, ' '), /a#b\.test\.mjs`?[^.]*\bkept\b[^.]*\ballowlist\b[^.]*\brefuses\b/,
|
||||||
|
'the doc says the allowlist still refuses the word');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('review 630 r8 / criteria-runner.mjs:890: no CLI compares process.argv[1] itself, and the three that did work through a symlinked path with a space', () => {
|
||||||
|
const files = spawnSync('git', ['ls-files', 'lib', 'scripts', 'hooks'], { cwd: ROOT, encoding: 'utf8' }).stdout.split('\n').filter((p) => p.endsWith('.mjs'));
|
||||||
|
assert.deepEqual(files.filter((p) => p !== 'lib/util/is-main.mjs' && /process\.argv\[1\]/.test(read(p))), [],
|
||||||
|
'only lib/util/is-main.mjs reads process.argv[1]');
|
||||||
|
const base = mkdtempSync(join(tmpdir(), 'r630 r8 main '));
|
||||||
|
const link = join(base, 'voyage link');
|
||||||
|
try {
|
||||||
|
symlinkSync(ROOT, link);
|
||||||
|
for (const cli of ['lib/verification/criteria-runner.mjs', 'scripts/comparison-gate.mjs', 'scripts/end-state-gate.mjs']) {
|
||||||
|
const r = spawnSync(process.execPath, [join(link, cli), '--no-such-flag'], { encoding: 'utf8', input: '', timeout: 30_000 });
|
||||||
|
assert.equal(r.status, 2, `${cli} through "${link}": ${r.stdout}${r.stderr}`);
|
||||||
|
assert.match(r.stderr, /unknown argument --no-such-flag/);
|
||||||
|
}
|
||||||
|
} finally { fs.rmSync(link, { force: true }); rmSync(base, { recursive: true, force: true }); }
|
||||||
|
});
|
||||||
|
|
||||||
|
test('review 630 r8 / plan-schema.mjs:353 + :322: a description beside a backticked path names no file; a build file in its own entry does', () => {
|
||||||
|
const step = (files) => `- **Files:** ${files}\n- **Changes:** x\n`;
|
||||||
|
// A description after a dash, or a word inside a sentence, is no path (code-review, code-correctness).
|
||||||
|
for (const files of ['`tests/lib/x.test.mjs` — mocks fs.readFileSync', '`tests/foo.test.mjs` — covers lib/foo.mjs',
|
||||||
|
'`tests/a.test.mjs`, runs on Node.js 22.x', '`tests/a.test.mjs` — exercises the parser in index.js',
|
||||||
|
'`tests/a.test.mjs`, no change to package.json', '`tests/a.test.mjs` - asserts JSON.parse behaviour',
|
||||||
|
'`tests/a.test.mjs`: covers src/a.mjs and Makefile']) {
|
||||||
|
assert.deepEqual(stepTestsOnly(step(files)), { testsOnly: true, files: ['tests/a.test.mjs', 'tests/foo.test.mjs', 'tests/lib/x.test.mjs'].filter((t) => files.startsWith(`\`${t}\``)) }, files);
|
||||||
|
}
|
||||||
|
// A file in its own entry, after "and", or a path with a slash before any description, still counts (round 7).
|
||||||
|
for (const [files, paths] of [
|
||||||
|
['`tests/a.test.mjs`, makefile', ['tests/a.test.mjs', 'makefile']],
|
||||||
|
['`tests/a.test.mjs`, justfile', ['tests/a.test.mjs', 'justfile']],
|
||||||
|
['`tests/a.test.mjs`, package.json', ['tests/a.test.mjs', 'package.json']],
|
||||||
|
['`tests/a.test.mjs` and package.json', ['tests/a.test.mjs', 'package.json']],
|
||||||
|
['`tests/a.test.mjs`, the CLI entry src/cli.mjs — reads argv', ['tests/a.test.mjs', 'src/cli.mjs']],
|
||||||
|
]) {
|
||||||
|
assert.deepEqual(stepTestsOnly(step(files)), { testsOnly: false, files: paths }, files);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('review 630 r8 / done-line.mjs:61 + test-lock.mjs:183: a stopped command is read by stoppedBy, the one reading', async () => {
|
||||||
|
const { writeDoneLine } = await import('../../lib/verification/done-line.mjs');
|
||||||
|
const repo = gitRepo('r630-r8-done-');
|
||||||
|
try {
|
||||||
|
writeFileSync(join(repo, 'README.md'), 'x\n');
|
||||||
|
commitAll(repo, 'base');
|
||||||
|
mkdirSync(join(repo, 'tests'));
|
||||||
|
writeFileSync(join(repo, 'tests', 'hang.test.mjs'), "import { test } from 'node:test';\ntest('hangs', () => new Promise((r) => setTimeout(r, 6000)));\n");
|
||||||
|
const r = writeDoneLine({ cwd: repo, step: 1, cmd: 'node --test tests/hang.test.mjs', report: join(repo, 'report.md'), timeoutMs: 2000 });
|
||||||
|
assert.equal(r.status, 'not-run', JSON.stringify(r));
|
||||||
|
assert.match(r.reason, /killed .*after the 2 s timeout/, r.reason);
|
||||||
|
} finally { rmSync(repo, { recursive: true, force: true }); }
|
||||||
|
for (const rel of ['lib/verification/done-line.mjs', 'lib/util/test-lock.mjs', 'lib/verification/red-proof.mjs']) {
|
||||||
|
const src = read(rel);
|
||||||
|
assert.match(src, /\bstoppedBy\(/, `${rel} reads a stop with stoppedBy`);
|
||||||
|
assert.doesNotMatch(src, /r\.error \|\| r\.status === null/, `${rel} does not read the spawn result by hand`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('review 630 r8 / red-proof.mjs:101: a run reports one reading of a stop, not a second spawnError beside it', async () => {
|
||||||
|
const { proveRed } = await import('../../lib/verification/red-proof.mjs');
|
||||||
|
const repo = gitRepo('r630-r8-spawn-');
|
||||||
|
try {
|
||||||
|
writeFileSync(join(repo, 'README.md'), 'x\n');
|
||||||
|
const base = commitAll(repo, 'base');
|
||||||
|
mkdirSync(join(repo, 'tests'));
|
||||||
|
writeFileSync(join(repo, 'tests', 'a.test.mjs'), 'x\n');
|
||||||
|
// `jest` is on the allowlist and not installed where this suite runs.
|
||||||
|
const r = proveRed({ cwd: repo, base, cmd: 'jest tests/a.test.mjs', tests: ['tests/a.test.mjs'], redOnly: true });
|
||||||
|
assert.equal(r.status, 'not-run', JSON.stringify(r));
|
||||||
|
assert.match(r.reason, /could not start/, r.reason);
|
||||||
|
const runs = [r.base, r.result].filter(Boolean);
|
||||||
|
assert.ok(runs.length > 0, JSON.stringify(r));
|
||||||
|
for (const run of runs) assert.ok(!('spawnError' in run), `no spawnError beside stopped: ${JSON.stringify(run)}`);
|
||||||
|
} finally { rmSync(repo, { recursive: true, force: true }); }
|
||||||
|
});
|
||||||
|
|
||||||
|
test('review 630 r8 / findings-schema.mjs:46: the block reader and the verdict gate agree on what a fence is', async () => {
|
||||||
|
const { extractFindingsBlock } = await import('../../lib/review/findings-schema.mjs');
|
||||||
|
const json = '{\n "findings": []\n}';
|
||||||
|
// Markdown: a closing fence may be longer than the opening one.
|
||||||
|
assert.equal(extractFindingsBlock(`x\n\n\`\`\`json\n${json}\n\`\`\`\`\n`), `${json}\n`, 'a ```` closer ends the block');
|
||||||
|
// Not a fence in Markdown, and not in the verdict gate either: three backticks
|
||||||
|
// after text on the same line.
|
||||||
|
assert.equal(extractFindingsBlock(`\`\`\`json\n${json}\`\`\`\n`), null, 'a closer after JSON on its line');
|
||||||
|
assert.equal(extractFindingsBlock(`see \`\`\`json\n${json}\n\`\`\`\n`), null, 'an opener after prose on its line');
|
||||||
|
const { decideVerdict } = await import('../../lib/review/verdict-gate.mjs');
|
||||||
|
const fm = ['---', 'type: trekreview', 'review_version: "1.1"', 'created: 2026-10-11', 'task: "t"', 'slug: s',
|
||||||
|
'project_dir: p/', 'brief_path: p/brief.md', `scope_sha_start: ${'b'.repeat(40)}`, `scope_sha_end: ${'c'.repeat(40)}`,
|
||||||
|
'reviewed_files_count: 1', 'verdict: WARN', 'findings: []', '---', ''].join('\n');
|
||||||
|
const payload = JSON.stringify({ verdict: 'WARN', counts: { BLOCKER: 0, MAJOR: 0, MINOR: 0, SUGGESTION: 0 }, allow_blocked_by: [], findings: [] }, null, 2);
|
||||||
|
for (const closer of ['```', '````']) {
|
||||||
|
const v = decideVerdict(`${fm}\n# R\n\n## Executive Summary\n\nx\n\n## Coverage\n\nx\n\n## Declined to Judge\n\nNone.\n\nTrimmed: 0\n\n` +
|
||||||
|
`## Remediation Summary\n\nx\n\n\`\`\`json\n${payload}\n${closer}\n`);
|
||||||
|
assert.equal(v.exit, 0, `closer ${closer}: ${v.reason}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('review 630 r8 / review-validator.mjs:36 + synthesis-digest-schema.mjs:59: three backticks in a JSON string do not end the block there either', async () => {
|
||||||
|
const { validateReviewContent } = await import('../../lib/validators/review-validator.mjs');
|
||||||
|
const { extractDigestBlock } = await import('../../lib/plan/synthesis-digest-schema.mjs');
|
||||||
|
const id = 'a'.repeat(40);
|
||||||
|
const payload = { verdict: 'BLOCK', counts: { BLOCKER: 1, MAJOR: 0, MINOR: 0, SUGGESTION: 0 }, allow_blocked_by: [],
|
||||||
|
findings: [{ id, severity: 'BLOCKER', rule_key: 'MISSING_TEST', file: 'a.mjs', line: 1, title: 'x',
|
||||||
|
detail: 'write the check as ```bash\nnpm test\n``` and run it', recommended_action: 'Add a test.' }] };
|
||||||
|
const review = ['---', 'type: trekreview', 'review_version: "1.1"', 'created: 2026-10-11', 'task: "t"', 'slug: s',
|
||||||
|
'project_dir: p/', 'brief_path: p/brief.md', 'scope_sha_start: ' + 'b'.repeat(40), 'scope_sha_end: ' + 'c'.repeat(40),
|
||||||
|
'reviewed_files_count: 1', 'verdict: BLOCK', 'findings:', ` - ${id}`, '---', '', '# Review', '',
|
||||||
|
'## Executive Summary', '', 'x', '', '## Coverage', '', 'x', '', '## Declined to Judge', '', 'None.', '', 'Trimmed: 0', '',
|
||||||
|
'## Remediation Summary', '', 'x', '', '```json', JSON.stringify(payload, null, 2), '```', ''].join('\n');
|
||||||
|
const r = validateReviewContent(review);
|
||||||
|
assert.ok([...r.errors, ...r.warnings].some((i) => i.code === 'REVIEW_MISSING_REMEDIATION_STEP'),
|
||||||
|
`a BLOCKER with no ### Step R<n> is found although its detail holds a fence: ${JSON.stringify(r.errors.map((e) => e.code))}`);
|
||||||
|
const digest = '```json\n{"note": "a ```bash fence"}\n```\n';
|
||||||
|
assert.equal(extractDigestBlock(digest), '{"note": "a ```bash fence"}\n');
|
||||||
|
});
|
||||||
|
|
|
||||||
|
|
@ -489,3 +489,26 @@ test('release-check r7 — a relative --review is read in --repo, like every oth
|
||||||
assert.equal(r.status, 0, r.stdout + r.stderr);
|
assert.equal(r.status, 0, r.stdout + r.stderr);
|
||||||
} finally { fx.cleanup(); }
|
} finally { fx.cleanup(); }
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// ==== round 8 of the release review of 6.3.0 (2026-10-11), fixed in 6.3.1 ====
|
||||||
|
|
||||||
|
test('release-check r8 — a relative --review written from the caller\'s directory, as 6.3.0 read it, still works → 0', () => {
|
||||||
|
const fx = fixture();
|
||||||
|
try {
|
||||||
|
// The review sits beside the repo (fx.base/review.md); run from fx.base.
|
||||||
|
const r = spawnSync(process.execPath, [CHECK, '--repo', 'repo', '--review', 'review.md'], { cwd: fx.base, encoding: 'utf8' });
|
||||||
|
assert.doesNotMatch(r.stdout, /does not exist/, r.stdout);
|
||||||
|
assert.equal(r.status, 0, r.stdout + r.stderr);
|
||||||
|
} finally { fx.cleanup(); }
|
||||||
|
});
|
||||||
|
|
||||||
|
test('release-check r8 — a relative --review that names two different files is not guessed → 2', () => {
|
||||||
|
const fx = fixture();
|
||||||
|
try {
|
||||||
|
writeFileSync(join(fx.repo, 'review.md'), readFileSync(fx.review, 'utf8'));
|
||||||
|
writeFileSync(join(fx.repo, '.git', 'info', 'exclude'), 'review.md\n');
|
||||||
|
const r = spawnSync(process.execPath, [CHECK, '--repo', 'repo', '--review', 'review.md'], { cwd: fx.base, encoding: 'utf8' });
|
||||||
|
assert.equal(r.status, 2, r.stdout + r.stderr);
|
||||||
|
assert.match(r.stdout, /two files/, r.stdout);
|
||||||
|
} finally { fx.cleanup(); }
|
||||||
|
});
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue