merge: the fixes from the release review of 6.3.0
Release review of v6.2.0..f669b87 (BLOCK). The BLOCKER was the review brief's wording of a non-goal (operator 2026-10-10: order 6's review loop is in 6.3.0); the hole behind it is closed (plan-validator --steps-from). The 12 MAJOR and 1 MINOR findings, and git --output in the criteria runner, are fixed. Suite 2148 (2146/0/2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
commit
0d044f6b6f
SSH key fingerprint:
SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
18 changed files with 450 additions and 43 deletions
|
|
@ -21,9 +21,10 @@ Voyage — a contract-driven Claude Code pipeline: brief, research, plan, execut
|
|||
> `tests/lib/no-claude-code-launch.test.mjs`; its exemption list holds historical
|
||||
> mentions plus one operator instrument (operator decision, 2026-10-02:
|
||||
> `docs/proevesett.md` tells a person how to run Anthropic's plugin eval tool on
|
||||
> the fixed test set; Voyage itself starts nothing) plus two files that only read
|
||||
> the session's permission mode from hook input (operator decision, 2026-10-06),
|
||||
> each with a reason and an exact count.
|
||||
> the fixed test set; Voyage itself starts nothing) plus four files that only read
|
||||
> the session's permission mode from hook input, two hooks and their tests
|
||||
> (operator decisions, 2026-10-06 and 2026-10-09), each with a reason and an
|
||||
> exact count.
|
||||
|
||||
## Commands
|
||||
|
||||
|
|
|
|||
|
|
@ -915,7 +915,13 @@ in this order:
|
|||
a merge when the branch is main), so that answer is the one question for the
|
||||
merge (operator decision 2026-10-08; it was two when `--gates` added a pause
|
||||
of its own). A declined guard question means the call did not run: stop and
|
||||
report it like `MERGE_REFUSED`, keeping `wave.json`.
|
||||
report it like `MERGE_REFUSED`, keeping `wave.json`. In the permission mode
|
||||
that shows no prompts the guard does not ask: it refuses the push and the
|
||||
merge (exit 2, "no one to ask"). Then stop, keep `wave.json`, and report that
|
||||
the guard refused because no one can be asked, not that local changes are in
|
||||
the way: the operator runs this session's push-and-merge block by hand, or
|
||||
switches to a mode that asks and re-runs. A re-run in the same mode is
|
||||
refused again.
|
||||
|
||||
**Merge session branches back (SEQUENTIAL, one at a time):**
|
||||
|
||||
|
|
@ -2216,9 +2222,12 @@ the run is `stopped`, and the merge stays closed.
|
|||
plan_version ≥ 1.7 plan the manifest the legacy rule synthesizes
|
||||
(Phase 2: `expected_paths` ← its Files, …): a rule already in this file, not
|
||||
a new design. Earlier steps are never touched.
|
||||
3. Check that the new steps can run, before anyone is asked:
|
||||
3. Check that the new steps can run, before anyone is asked. `--steps-from`
|
||||
holds the appended steps to the step rules (`Test first:` with its `File:`,
|
||||
one runnable `Verify:`) whatever the plan's version; below 1.8 nothing else
|
||||
would:
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/plan-validator.mjs --strict --json "{plan_path}"
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/plan-validator.mjs --strict --json --steps-from {total + 1} "{plan_path}"
|
||||
```
|
||||
Any exit but 0 (a `Verify:` that is not one test-runner call, a
|
||||
`Test first:` without its `File:`, …): restore the plan from the copy,
|
||||
|
|
|
|||
|
|
@ -755,7 +755,9 @@ Write the plan following the template structure. The plan MUST include:
|
|||
`End-to-end: \`bash tests/e2e.sh\``. The same command is also listed under
|
||||
`## Verification`: Phase 7 of `/trekexecute` runs that section, and a passed
|
||||
criterion that runs the step's `End-to-end:` command is what verifies the
|
||||
run end to end.
|
||||
run end to end. `End-to-end:` rides on the last step that builds behaviour,
|
||||
with `tests/e2e.sh` in that step's `Files:`: a step of its own that only adds
|
||||
the script is a step of only tests, and it can never be proven red.
|
||||
10. **Execution Strategy** — For plans with > 5 steps: group steps into sessions
|
||||
(3–5 steps each), organize sessions into waves (parallel where independent),
|
||||
specify scope fences per session. Omit for plans with ≤ 5 steps.
|
||||
|
|
|
|||
|
|
@ -423,7 +423,9 @@ same stable error codes (`FINDINGS_NO_JSON_BLOCK`, `FINDINGS_PARSE_ERROR`,
|
|||
`FINDING_PROSE_LINE_OUT_OF_RANGE` is the same check on a `path:line` written
|
||||
in prose. In a finding's own fields (`title`, `detail`, `recommended_action`,
|
||||
`brief_ref`) it is an **error**: re-ask that reviewer like any other schema
|
||||
failure. In the prose around the JSON block it is a **warning**: the re-ask
|
||||
failure, and quote the way a removed line is cited, `<rev>:<path>:<line>`
|
||||
(`v6.2.0:hooks/x.mjs:120`, not judged against today's file), so a finding
|
||||
about deleted code keeps its line instead of losing it. In the prose around the JSON block it is a **warning**: the re-ask
|
||||
asks for the block only and cannot correct that prose, so do not re-ask —
|
||||
report each such warning (reviewer, citation, the file's line count) in the
|
||||
Coverage section of `review.md`.
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ import { readFileSync } from 'node:fs';
|
|||
// The guard's own modules are loaded so that a broken install cannot crash
|
||||
// the hook: a crash (exit 1) is a non-blocking error to Claude Code, which
|
||||
// would let every command through. A module that will not load makes the
|
||||
// guard ask about every command instead.
|
||||
// guard ask about every command instead, and refuse it where no one can be asked.
|
||||
let lib;
|
||||
let loadError = null;
|
||||
try {
|
||||
|
|
@ -217,7 +217,22 @@ if (!command || typeof command !== 'string') {
|
|||
process.exit(0);
|
||||
}
|
||||
|
||||
// In the permission mode where Claude Code shows no prompts, an "ask" reaches
|
||||
// no one and the call runs unattended (see the refusal below).
|
||||
const NO_PROMPT_MODE = 'bypassPermissions';
|
||||
const noOneToAsk = input.permission_mode === NO_PROMPT_MODE;
|
||||
|
||||
if (loadError) {
|
||||
// A guard that cannot read its rules asks about every command; where no one
|
||||
// can be asked it refuses instead (release review of 6.3.0, 2026-10-10).
|
||||
if (noOneToAsk) {
|
||||
process.stderr.write(
|
||||
`[voyage] BLOCKED: Voyage's Bash guard could not load its rules (${loadError.message}) and there is no one to ask\n` +
|
||||
' This session runs in the permission mode where Claude Code asks nothing, so the guard cannot ask a person. ' +
|
||||
'Way forward: reinstall Voyage to restore the guard, or run the command yourself.\n'
|
||||
);
|
||||
process.exit(2);
|
||||
}
|
||||
process.stdout.write(JSON.stringify({
|
||||
hookSpecificOutput: {
|
||||
hookEventName: 'PreToolUse',
|
||||
|
|
@ -280,8 +295,7 @@ try {
|
|||
// decisions 2026-10-09: first merge and push, then the other two classes).
|
||||
// /trekexecute relies on this: its main-merge gate is a lock, and this guard's
|
||||
// question is the one question.
|
||||
const NO_PROMPT_MODE = 'bypassPermissions';
|
||||
const unattended = input.permission_mode === NO_PROMPT_MODE ? approvals : [];
|
||||
const unattended = noOneToAsk ? approvals : [];
|
||||
if (unattended.length > 0) {
|
||||
process.stderr.write(
|
||||
`[voyage] BLOCKED: ${unattended.map((m) => m.name).join(', ')} with no one to ask\n` +
|
||||
|
|
|
|||
|
|
@ -288,10 +288,12 @@ export function onFailureAction(text) {
|
|||
* The paths a field names. Each entry (comma- or line-separated, a sub-bullet's
|
||||
* `-` dropped) gives its backticked spans, or else its first word: a plain
|
||||
* path beside a backticked one is still a path (independent review, order 10).
|
||||
* A parenthesised note is not an entry: "`a.test.mjs` (new: covers x, y)"
|
||||
* names one path, not "y)" too (release review of 6.3.0, 2026-10-10).
|
||||
*/
|
||||
function fieldPaths(text) {
|
||||
const out = [];
|
||||
for (const raw of String(text ?? '').split(/[,\n]/)) {
|
||||
for (const raw of String(text ?? '').replace(/\([^()\n]*\)/g, ' ').split(/[,\n]/)) {
|
||||
const entry = raw.trim().replace(/^[-*]\s+/, '');
|
||||
if (entry === '') continue;
|
||||
const spans = [...entry.matchAll(/`([^`\n]+)`/g)].map((m) => m[1].trim());
|
||||
|
|
@ -303,7 +305,11 @@ function fieldPaths(text) {
|
|||
|
||||
// A document is never the test here: `docs/specs/api.md` sits under a `specs/`
|
||||
// folder, which lib/util/test-path.mjs reads as tests for the floor guard.
|
||||
// Nor is a data file under a `spec/` or `specs/` folder (`api/specs/openapi.yaml`
|
||||
// is an API spec); under tests/ it is a fixture, and a fixture is a test file.
|
||||
const DOC_RE = /\.(md|markdown|mdx|txt|rst|adoc)$/i;
|
||||
const DATA_RE = /\.(ya?ml|json|toml|xml|csv)$/i;
|
||||
const UNDER_TESTS_RE = /(^|\/)(tests?|__tests__)\//;
|
||||
|
||||
/**
|
||||
* Whether every file in a step's `Files:` is a test: the step's own `Test
|
||||
|
|
@ -324,7 +330,8 @@ export function stepTestsOnly(body) {
|
|||
const testFirst = f.has('Test first')
|
||||
? [...f.get('Test first').text.matchAll(/\bFile:\**[ \t]*`([^`\n]+)`/gi)].map((m) => m[1].trim())
|
||||
: [];
|
||||
const isTest = (p) => testFirst.includes(p) || (isTestPath(p) && !DOC_RE.test(p));
|
||||
const isTest = (p) => testFirst.includes(p) ||
|
||||
(isTestPath(p) && !DOC_RE.test(p) && !(DATA_RE.test(p) && !UNDER_TESTS_RE.test(p)));
|
||||
return { testsOnly: files.length > 0 && files.every(isTest), files };
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -162,7 +162,9 @@ function checkProseCitations(text, where, repoRoot, errors, seen) {
|
|||
errors.push(issue(
|
||||
'FINDING_PROSE_LINE_OUT_OF_RANGE',
|
||||
`${where} cites ${m[0]}, but ${m[1]} has ${n} lines`,
|
||||
'Cite the line number in the file itself (Read it), not the line in the diff or patch, or drop the line number.',
|
||||
'Cite the line number in the file itself (Read it), not the line in the diff or patch. A line that is no ' +
|
||||
'longer in the file is cited with its revision, `<rev>:<path>:<line>` (`v6.2.0:hooks/x.mjs:120`), which ' +
|
||||
'is not judged against today\'s file.',
|
||||
));
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -145,7 +145,8 @@ export function lintSkill(skillDir, opts = {}) {
|
|||
if (/lint-exempt/i.test(text)) errors.push(err('SKILL_SELF_EXEMPTION', 'a skill may not exempt itself; exemptions live in EXEMPTIONS in lib/skills/skill-lint.mjs'));
|
||||
|
||||
// Size
|
||||
const lineCount = text.split('\n').length;
|
||||
// A file that ends in a newline has no line after it (findings-schema counts the same way).
|
||||
const lineCount = text.endsWith('\n') ? text.split('\n').length - 1 : text.split('\n').length;
|
||||
if (lineCount >= MAX_LINES) errors.push(err('SKILL_TOO_LONG', `SKILL.md has ${lineCount} lines (must stay under ${MAX_LINES}); move detail to a linked file`));
|
||||
|
||||
// Body: headings, enforcement, references, shell injection
|
||||
|
|
|
|||
|
|
@ -164,13 +164,19 @@ function checkRecovery(sections, opts) {
|
|||
`Open with one of ${ON_FAILURE_ACTIONS.join(' | ')}, then the specifics (\`retry once, then escalate\`).`));
|
||||
}
|
||||
if (stepTestsOnly(s.body).testsOnly) {
|
||||
// The run-the-thing script alone is such a step too: `End-to-end:` rides on
|
||||
// the last step that builds behaviour (release review of 6.3.0, 2026-10-10).
|
||||
const e2e = f.has('End-to-end')
|
||||
? ' `End-to-end:` rides on the last step that builds behaviour, with its script in that step\'s `Files:`, ' +
|
||||
'not on a step of its own.'
|
||||
: '';
|
||||
warnings.push(issue('PLAN_STEP_TESTS_ONLY',
|
||||
`Step ${s.n}: every file in \`Files:\` is a test; a step of only tests can never be proven red`,
|
||||
'The test for a behaviour belongs in the step that builds it: at this step\'s start point the behaviour ' +
|
||||
'already exists (an earlier step built it, or it was there before the plan), so its test passes there and ' +
|
||||
'red proof rejects it (`not-red-at-base`). Move the test into the step that builds the behaviour; a test ' +
|
||||
'meant to pass from the start (a golden or characterization test) is the `Regression:` of the step that ' +
|
||||
'changes the code, not a step of its own.'));
|
||||
'changes the code, not a step of its own.' + e2e));
|
||||
}
|
||||
}
|
||||
return { errors, warnings };
|
||||
|
|
@ -286,6 +292,27 @@ export function validatePlanContent(rawText, opts = {}) {
|
|||
else warnings.push(...recovery.errors);
|
||||
warnings.push(...recovery.warnings);
|
||||
|
||||
// Steps a run appended (the review loop's remediation steps, /trekexecute
|
||||
// Phase 7.2) must be runnable steps whatever the plan's version: below 1.8
|
||||
// the step rules never ran, and a review step whose Verify is "re-run
|
||||
// /trekreview" validated and was executed (release review of 6.3.0,
|
||||
// 2026-10-10). From 1.8 the rules below already cover every step.
|
||||
const stepsFrom = Number.isInteger(opts.stepsFrom) && opts.stepsFrom > 0 ? opts.stepsFrom : null;
|
||||
if (stepsFrom !== null && !versionAtLeast(planVersion, '1.8')) {
|
||||
const appended = sections.filter((s) => Number(s.n) >= stepsFrom);
|
||||
const stepIssues = [
|
||||
...checkSteps(appended).filter((i) => i.code !== 'PLAN_NO_END_TO_END'),
|
||||
...checkRunnable('', appended, { cwd: opts.cwd }),
|
||||
];
|
||||
for (const s of appended) {
|
||||
if (!parseStepFields(s.body).has('Verify')) {
|
||||
stepIssues.push(issue('PLAN_STEP_VERIFY_NOT_RUNNABLE', `Step ${s.n}: no \`Verify:\` field`,
|
||||
'An appended step needs one test command: proof of red and the done line run it.'));
|
||||
}
|
||||
}
|
||||
errors.push(...stepIssues);
|
||||
}
|
||||
|
||||
const ceremony = extractPlanCeremony(text);
|
||||
if (versionAtLeast(planVersion, '1.8')) {
|
||||
const v18 = [...checkSteps(sections), ...checkRunnable(text, sections, { cwd: opts.cwd })];
|
||||
|
|
@ -352,12 +379,14 @@ function isMain() {
|
|||
if (isMain()) {
|
||||
const args = process.argv.slice(2);
|
||||
const strict = !args.includes('--soft');
|
||||
const filePath = args.find(a => !a.startsWith('--'));
|
||||
if (!filePath) {
|
||||
process.stderr.write('Usage: plan-validator.mjs [--strict|--soft] <plan.md>\n');
|
||||
const at = args.indexOf('--steps-from');
|
||||
const stepsFrom = at >= 0 ? Number(args[at + 1]) : null;
|
||||
const filePath = args.find((a, i) => !a.startsWith('--') && !(at >= 0 && i === at + 1));
|
||||
if (!filePath || (at >= 0 && !(Number.isInteger(stepsFrom) && stepsFrom > 0))) {
|
||||
process.stderr.write('Usage: plan-validator.mjs [--strict|--soft] [--json] [--steps-from <N>] <plan.md>\n');
|
||||
process.exit(2);
|
||||
}
|
||||
const r = validatePlan(filePath, { strict });
|
||||
const r = validatePlan(filePath, { strict, stepsFrom });
|
||||
if (args.includes('--json')) {
|
||||
process.stdout.write(JSON.stringify({
|
||||
valid: r.valid,
|
||||
|
|
|
|||
|
|
@ -315,10 +315,13 @@ const RUNNERS = {
|
|||
bash: (t) => (isTestScript(t[1]) ? permit() : deny('bash, but not a script under tests/')),
|
||||
// The system bash by its path (macOS: 3.2), so a check can pin the shell.
|
||||
'/bin/bash': (t) => (isTestScript(t[1]) ? permit() : deny('/bin/bash, but not a script under tests/')),
|
||||
git: (t) =>
|
||||
READONLY_GIT.has(t[1])
|
||||
? permit()
|
||||
: deny(`git, but not a read-only subcommand first (${[...READONLY_GIT].join('|')})`),
|
||||
git: (t) => {
|
||||
if (!READONLY_GIT.has(t[1])) return deny(`git, but not a read-only subcommand first (${[...READONLY_GIT].join('|')})`);
|
||||
// `--output` makes diff/log/show write their output to a file (release
|
||||
// review of 6.3.0, 2026-10-10: a plan's check could overwrite any file).
|
||||
if (t.some((w) => w === '--output' || w.startsWith('--output='))) return deny('git with --output writes a file, so it is not a read-only check');
|
||||
return permit();
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -24,23 +24,23 @@
|
|||
// progress.json uses (C15). Nothing else is translated.
|
||||
//
|
||||
// The command runs the way the runner runs it (`/bin/sh -c`), but without
|
||||
// NODE_TEST_CONTEXT in its environment (red-proof.mjs `commandEnv`): measured
|
||||
// NODE_TEST_CONTEXT in its environment (lib/util/command-env.mjs): measured
|
||||
// 2026-10-08, a red `node --test` regression read as passed when this CLI was
|
||||
// itself started under `node --test`, because the nested run reported to the
|
||||
// parent and exited 0.
|
||||
|
||||
function exec(command, { cwd, timeoutMs }) {
|
||||
const r = spawnSync('/bin/sh', ['-c', command], { cwd, encoding: 'utf8', timeout: timeoutMs, env: commandEnv() });
|
||||
return { status: r.status, stdout: r.stdout ?? '', stderr: r.stderr ?? '' };
|
||||
}
|
||||
|
||||
import { readFileSync, realpathSync } from 'node:fs';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { sliceSteps, parseStepFields } from '../parsers/plan-schema.mjs';
|
||||
import { firstCommand, runCriteria } from './criteria-runner.mjs';
|
||||
import { commandEnv } from './red-proof.mjs';
|
||||
import { commandEnv } from '../util/command-env.mjs';
|
||||
|
||||
function exec(command, { cwd, timeoutMs }) {
|
||||
const r = spawnSync('/bin/sh', ['-c', command], { cwd, encoding: 'utf8', timeout: timeoutMs, env: commandEnv() });
|
||||
return { status: r.status, stdout: r.stdout ?? '', stderr: r.stderr ?? '' };
|
||||
}
|
||||
|
||||
export const REGRESSION_FIELD = 'Regression';
|
||||
const EXIT = Object.freeze({ passed: 0, absent: 0, failed: 1, blocked: 1, unrunnable: 1, error: 2 });
|
||||
|
|
|
|||
|
|
@ -19,7 +19,8 @@
|
|||
// stops (1), undecided stops (2)
|
||||
// coverage the review is of THIS history: its `scope_sha_end` is HEAD or an
|
||||
// ancestor of HEAD, and the commits after it touch only the release
|
||||
// bookkeeping (version files, CHANGELOG, the README version badge)
|
||||
// bookkeeping (the CHANGELOG, the version in the version files,
|
||||
// the README version badge)
|
||||
// door the CHANGELOG section for the version has a `### Doors` list with
|
||||
// one `- one-way: …` or `- two-way: …` line per delivery
|
||||
//
|
||||
|
|
@ -37,6 +38,7 @@ import { spawnSync } from 'node:child_process';
|
|||
import { dirname, join, relative, resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { decideVerdict } from '../lib/review/verdict-gate.mjs';
|
||||
import { commandEnv } from '../lib/util/command-env.mjs';
|
||||
import { parseDocument } from '../lib/util/frontmatter.mjs';
|
||||
|
||||
const PLUGIN_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
|
||||
|
|
@ -44,12 +46,27 @@ const PLUGIN_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
|
|||
export const EXIT = Object.freeze({ READY: 0, NOT_READY: 1, UNDECIDED: 2 });
|
||||
export const CHECK_IDS = Object.freeze(['tree', 'suite', 'review', 'coverage', 'door']);
|
||||
|
||||
// Files a release commit may change after the reviewed commit. README.md is
|
||||
// allowed only for lines that carry the version badge.
|
||||
// Files a release commit may change after the reviewed commit, and how much of
|
||||
// each (release review of 6.3.0, 2026-10-10: a changed test script or hook
|
||||
// wiring passed as bookkeeping): the CHANGELOG freely; the version files only
|
||||
// in their version (the JSON with it taken out must be equal); README.md only
|
||||
// on the lines that carry the version badge.
|
||||
export const RELEASE_BOOKKEEPING = Object.freeze([
|
||||
'CHANGELOG.md', 'package.json', 'package-lock.json', '.claude-plugin/plugin.json', 'README.md',
|
||||
]);
|
||||
const BADGE_RE = /badge\/version-/;
|
||||
const VERSION_FILES = Object.freeze(['package.json', 'package-lock.json', '.claude-plugin/plugin.json']);
|
||||
|
||||
// The JSON of a version file with its version taken out: the top-level
|
||||
// `version`, and in a lockfile the root package's `packages[""].version`.
|
||||
export function withoutVersion(text) {
|
||||
const o = JSON.parse(text);
|
||||
if (o && typeof o === 'object') {
|
||||
delete o.version;
|
||||
if (o.packages && o.packages[''] && typeof o.packages[''] === 'object') delete o.packages[''].version;
|
||||
}
|
||||
return JSON.stringify(o);
|
||||
}
|
||||
const DOOR_LINE_RE = /^- (one-way|two-way): \S/;
|
||||
|
||||
const git = (cwd, args) => spawnSync('git', args, { cwd, encoding: 'utf8' });
|
||||
|
|
@ -100,6 +117,19 @@ export function parseDoors(section) {
|
|||
return { found: true, entries, bad };
|
||||
}
|
||||
|
||||
// The +/- lines of a unified diff, read inside its hunks only: a content line
|
||||
// that starts with "++ " or "-- " looks like a file header but is not one.
|
||||
export function changedLines(diff) {
|
||||
const out = [];
|
||||
let inHunk = false;
|
||||
for (const l of diff.split('\n')) {
|
||||
if (l.startsWith('diff --git ')) inHunk = false;
|
||||
else if (l.startsWith('@@')) inHunk = true;
|
||||
else if (inHunk && /^[+-]/.test(l)) out.push(l);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function checkTree(repo) {
|
||||
const r = git(repo, ['status', '--porcelain']);
|
||||
if (r.status !== 0) return { id: 'tree', status: 'error', detail: `git status failed: ${r.stderr.trim()}` };
|
||||
|
|
@ -112,7 +142,8 @@ function checkTree(repo) {
|
|||
|
||||
function checkSuite(repo, treeClean) {
|
||||
if (!treeClean) return { id: 'suite', status: 'fail', detail: 'suite not run: the tree has uncommitted changes, so it would not measure HEAD' };
|
||||
const r = spawnSync('npm', ['test'], { cwd: repo, encoding: 'utf8', maxBuffer: 256 * 1024 * 1024 });
|
||||
// commandEnv: under a parent `node --test` the suite would report to it and exit 0 whatever it found.
|
||||
const r = spawnSync('npm', ['test'], { cwd: repo, encoding: 'utf8', maxBuffer: 256 * 1024 * 1024, env: commandEnv() });
|
||||
if (r.error) return { id: 'suite', status: 'error', detail: `npm test could not start: ${r.error.message} — not measured` };
|
||||
const out = `${r.stdout ?? ''}\n${r.stderr ?? ''}`;
|
||||
const summary = out.split('\n').filter((l) => /^ℹ (tests|pass|fail|skipped) /.test(l)).map((l) => l.slice(2));
|
||||
|
|
@ -150,10 +181,20 @@ function checkCoverage(repo, reviewText) {
|
|||
if (names.status !== 0) return { id: 'coverage', status: 'error', detail: `git diff failed: ${names.stderr.trim()}` };
|
||||
const changed = names.stdout.split('\n').filter(Boolean);
|
||||
const outside = changed.filter((f) => !RELEASE_BOOKKEEPING.includes(f));
|
||||
for (const file of changed.filter((f) => VERSION_FILES.includes(f))) {
|
||||
const before = git(repo, ['show', `${full}:${file}`]);
|
||||
const after = git(repo, ['show', `HEAD:${file}`]);
|
||||
if (before.status !== 0 || after.status !== 0) { outside.push(`${file} (added or removed after the review)`); continue; }
|
||||
let same;
|
||||
try { same = withoutVersion(before.stdout) === withoutVersion(after.stdout); } catch (e) {
|
||||
return { id: 'coverage', status: 'error', detail: `${file} is not JSON (${e.message}) — not measured` };
|
||||
}
|
||||
if (!same) outside.push(`${file} (lines other than the version)`);
|
||||
}
|
||||
if (changed.includes('README.md')) {
|
||||
const d = git(repo, ['diff', '-U0', full, 'HEAD', '--', 'README.md']);
|
||||
const moved = d.stdout.split('\n').filter((l) => /^[+-]/.test(l) && !/^(\+\+\+|---)( |$)/.test(l));
|
||||
if (moved.some((l) => !BADGE_RE.test(l))) outside.push('README.md (lines other than the version badge)');
|
||||
if (d.status !== 0) return { id: 'coverage', status: 'error', detail: `git diff of README.md failed: ${d.stderr.trim()} — not measured` };
|
||||
if (changedLines(d.stdout).some((l) => !BADGE_RE.test(l))) outside.push('README.md (lines other than the version badge)');
|
||||
}
|
||||
if (outside.length > 0) {
|
||||
return { id: 'coverage', status: 'fail', detail: `changed after the reviewed commit ${full.slice(0, 12)} and not covered by the review: ${outside.join(', ')}`, items: outside };
|
||||
|
|
|
|||
|
|
@ -75,7 +75,8 @@ own. Preparatory refactoring comes first, as its own steps. Every step has
|
|||
end to end (`End-to-end:`). The test for a behaviour belongs in the step that
|
||||
builds it: a step of only tests for behaviour an earlier step delivered passes
|
||||
at its start point and can never be proven red (the validator warns:
|
||||
`PLAN_STEP_TESTS_ONLY`).
|
||||
`PLAN_STEP_TESTS_ONLY`). So `End-to-end:` rides on the last step that builds
|
||||
behaviour, with its script in that step's `Files:`, not on a step of its own.
|
||||
|
||||
Optional per-step fields: `Interface in:` (what the step consumes, and which
|
||||
earlier step or existing code produces it), `Interface out:` (what it produces
|
||||
|
|
|
|||
|
|
@ -265,7 +265,7 @@ test('P11: a review step in the template form is caught by the plan validator, s
|
|||
assert.ok(errors.some((c) => /VERIFY_NOT_RUNNABLE|TEST_FIRST/.test(c)), errors.join(','));
|
||||
|
||||
const p = section(CMD, '## Phase 7.2 — Simplify, review, verdict (P11)');
|
||||
const check = p.indexOf('plan-validator.mjs --strict --json "{plan_path}"');
|
||||
const check = p.indexOf('plan-validator.mjs --strict --json --steps-from {total + 1} "{plan_path}"');
|
||||
const ask = p.indexOf('Approve the plan as it is now?');
|
||||
assert.ok(check > 0 && ask > check, 'the validator runs before the approval question');
|
||||
assert.match(p, /restore the plan from the copy/, 'a stop leaves the plan matching its approval');
|
||||
|
|
@ -332,6 +332,56 @@ test('P12: the guard asks before a merge into main; in the mode with no prompts
|
|||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
// Release review of 6.3.0 (2026-10-10): the refusal covers the guard's two
|
||||
// fallbacks too — rules that will not load, and a command it could not read.
|
||||
function guardIn(plugin, cwd, command, permission_mode) {
|
||||
return node([join(plugin, 'hooks', 'scripts', 'pre-bash-executor.mjs')], {
|
||||
input: JSON.stringify({ tool_name: 'Bash', tool_input: { command }, cwd, permission_mode, session_id: 'coupling', tool_use_id: `t${Math.random()}` }),
|
||||
});
|
||||
}
|
||||
|
||||
test('P12 / review 630: a guard whose rules will not load refuses where no one is asked', async () => {
|
||||
const { pluginCopy } = await import('../hooks/guard-helpers.mjs');
|
||||
const plugin = pluginCopy({ 'hooks/lib/git-approval.mjs': 'missing' });
|
||||
const dir = repo();
|
||||
try {
|
||||
const asked = guardIn(plugin, dir, 'git status', 'default');
|
||||
assert.equal(asked.status, 0, asked.stderr);
|
||||
assert.match(asked.stdout, /"permissionDecision":"ask"/, 'the control: where someone can answer, the broken guard asks');
|
||||
const r = guardIn(plugin, dir, 'git push origin main', NO_PROMPT_MODE);
|
||||
assert.equal(r.status, 2, `refused, not asked: ${r.stdout}${r.stderr}`);
|
||||
assert.match(r.stderr, /no one to ask/);
|
||||
assert.match(r.stderr, /could not load/);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
rmSync(plugin, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('P12 / review 630: a command the classifier could not read is asked, and refused where no one is asked', async () => {
|
||||
const { pluginCopy } = await import('../hooks/guard-helpers.mjs');
|
||||
const plugin = pluginCopy();
|
||||
const lib = join(plugin, 'hooks', 'lib', 'git-approval.mjs');
|
||||
const src = readFileSync(lib, 'utf8');
|
||||
assert.ok(src.includes('export function classifyGitApproval('), 'the copy carries the classifier');
|
||||
writeFileSync(lib, src.replace('export function classifyGitApproval(',
|
||||
'export function classifyGitApproval() { throw new Error(\'classifier broke\'); }\nfunction classifyGitApprovalUnused('));
|
||||
const dir = repo();
|
||||
try {
|
||||
const asked = guardIn(plugin, dir, 'git push origin feature', 'default');
|
||||
assert.equal(asked.status, 0, asked.stderr);
|
||||
assert.match(asked.stdout, /"permissionDecision":"ask"/, 'where someone can answer it is asked');
|
||||
assert.match(asked.stdout, /could not read/);
|
||||
const r = guardIn(plugin, dir, 'git push origin feature', NO_PROMPT_MODE);
|
||||
assert.equal(r.status, 2, `refused: ${r.stdout}${r.stderr}`);
|
||||
assert.match(r.stderr, /could not read/);
|
||||
assert.match(r.stderr, /no one to ask/);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
rmSync(plugin, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// ---- the other new progress fields (Handover 5, additive-optional) -----------
|
||||
|
||||
test('H5: the wave-2 fields are shape-checked when present and optional when absent', () => {
|
||||
|
|
|
|||
2
tests/fixtures/jsonl-schemas.md
vendored
2
tests/fixtures/jsonl-schemas.md
vendored
|
|
@ -22,7 +22,7 @@
|
|||
| trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason | commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) | trekbrief.md:1130-1145 | profile, phase_models, profile_source | none |
|
||||
| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | lib/stats/stats-append.mjs (trekresearch.md Stats tracking) | trekresearch.md:782-808 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
|
||||
| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | lib/stats/stats-append.mjs (trekplan.md Phase 12) | trekplan.md:1115-1135 | profile, phase_models, parallel_agents, profile_source | none |
|
||||
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:2858-2880 | profile, phase_models, profile_source | none |
|
||||
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:2867-2889 | profile, phase_models, profile_source | none |
|
||||
| trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:78-99 | payload.profile, payload.phase_models, payload.profile_source | none |
|
||||
| trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) |
|
||||
| trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | lib/stats/stats-append.mjs (trekreview.md Phase 8) | trekreview.md:628-632 | profile, phase_models, profile_source | none |
|
||||
|
|
|
|||
|
|
@ -109,11 +109,12 @@ test('review 8: a command nobody was asked about is not logged, even if it reads
|
|||
// Operator decision 2026-10-09: where no person can be asked, the guard refuses
|
||||
// a merge into main or a push (exit 2) instead of asking, so the call never runs
|
||||
// and nothing reaches the approval log. post-approval-log's own check of the
|
||||
// recorded mode stays as a second line.
|
||||
// recorded mode stays as a second line (its own test below).
|
||||
const NO_PROMPT_MODE = 'bypassPermissions';
|
||||
test('uncertain (no-prompt mode): the push to main is refused, and no approval is logged', async () => {
|
||||
const plugin = pluginCopy({ 'lib/util/approval-log.mjs': 'stub' });
|
||||
const repo = makeRepo('main');
|
||||
const base = { session_id: `s-b-${RUN}`, tool_use_id: 'toolu_b', cwd: repo, tool_name: 'Bash', tool_input: { command: 'git push origin main' }, permission_mode: 'bypassPermissions' };
|
||||
const base = { session_id: `s-b-${RUN}`, tool_use_id: 'toolu_b', cwd: repo, tool_name: 'Bash', tool_input: { command: 'git push origin main' }, permission_mode: NO_PROMPT_MODE };
|
||||
const pre = await runHook(join(plugin, 'hooks', 'scripts', 'pre-bash-executor.mjs'), { ...base, hook_event_name: 'PreToolUse' });
|
||||
assert.equal(pre.code, 2, `refused, not asked: ${JSON.stringify(pre)}`);
|
||||
const log = join(tmp(), 'stub.jsonl');
|
||||
|
|
@ -121,6 +122,25 @@ test('uncertain (no-prompt mode): the push to main is refused, and no approval i
|
|||
assert.equal(readStubLog(log).length, 0);
|
||||
});
|
||||
|
||||
// The second line on its own (release review of 6.3.0, 2026-10-10): a record
|
||||
// that says no person was asked is never logged, whoever wrote it.
|
||||
test('uncertain (no-prompt mode): a recorded question no one saw is not logged', async () => {
|
||||
const plugin = pluginCopy({ 'lib/util/approval-log.mjs': 'stub' });
|
||||
const repo = makeRepo('main');
|
||||
const { writeApprovalRecord } = await import(join(plugin, 'hooks', 'lib', 'approval-record.mjs'));
|
||||
const ids = { session_id: `s-r-${RUN}`, tool_use_id: 'toolu_r' };
|
||||
assert.ok(writeApprovalRecord(ids.session_id, ids.tool_use_id, {
|
||||
command: 'git push origin main', permission_mode: NO_PROMPT_MODE,
|
||||
matches: [{ kind: 'merge', name: 'push to main', subject: repo }], asked_at: new Date().toISOString(),
|
||||
}), 'the record is written');
|
||||
const log = join(tmp(), 'stub.jsonl');
|
||||
const r = await runHookWithEnv(join(plugin, 'hooks', 'scripts', 'post-approval-log.mjs'), {
|
||||
...ids, hook_event_name: 'PostToolUse', cwd: repo, tool_name: 'Bash', tool_input: { command: 'git push origin main' },
|
||||
}, { STUB_LOG: log });
|
||||
assert.match(parseJsonOut(r.stdout).hookSpecificOutput.additionalContext, /not logged/i);
|
||||
assert.equal(readStubLog(log).length, 0);
|
||||
});
|
||||
|
||||
// 9. Classification
|
||||
const kinds = (cmd, branch = 'feat', cwd) => classifyGitApproval(cmd, { branch, cwd }).map((m) => m.kind);
|
||||
for (const cmd of ['git tag -v v1', 'git tag --verify v1', 'git push -n', 'git push --dry-run origin main']) {
|
||||
|
|
|
|||
166
tests/lib/review-630.test.mjs
Normal file
166
tests/lib/review-630.test.mjs
Normal file
|
|
@ -0,0 +1,166 @@
|
|||
// tests/lib/review-630.test.mjs
|
||||
// The findings of the release review of 6.3.0 (2026-10-10), each held by a
|
||||
// test that failed on f669b87. The guard's and the release check's own tests
|
||||
// live next to their files (tests/commands/trekexecute-coupling.test.mjs,
|
||||
// tests/hooks/guards-review-regressions.test.mjs, tests/scripts/release-check.test.mjs).
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { cpSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { buildPlan } from '../helpers/plan-v18-builder.mjs';
|
||||
import { validatePlanContent } from '../../lib/validators/plan-validator.mjs';
|
||||
import { stepTestsOnly } from '../../lib/parsers/plan-schema.mjs';
|
||||
import { validateFindings } from '../../lib/review/findings-schema.mjs';
|
||||
import { lintSkill } from '../../lib/skills/skill-lint.mjs';
|
||||
|
||||
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
||||
const read = (rel) => readFileSync(join(ROOT, rel), 'utf8');
|
||||
const VALIDATOR = join(ROOT, 'lib', 'validators', 'plan-validator.mjs');
|
||||
|
||||
// The review template's own Step R1, appended the Phase 7.2 way.
|
||||
function withTemplateStep(plan, n) {
|
||||
const r1 = /### Step R1:[^\n]*\n[\s\S]*?(?=\n> )/.exec(read('templates/trekreview-template.md'));
|
||||
assert.ok(r1, 'templates/trekreview-template.md carries a ### Step R1');
|
||||
// Phase 7.2 step 2 also adds the manifest the legacy rule synthesizes (expected_paths ← Files).
|
||||
const appended = r1[0].replace(/^### Step R1:/, `### Step ${n}:`) +
|
||||
'- **Checkpoint:** `git commit -m "fix(review): x [01234567]"`\n' +
|
||||
'- **Manifest:**\n ```yaml\n manifest:\n expected_paths:\n - lib/foo.mjs\n min_file_count: 1\n' +
|
||||
' commit_message_pattern: "^fix\\\\(review\\\\):"\n bash_syntax_check: []\n forbidden_paths: []\n must_contain: []\n ```\n';
|
||||
return plan.replace('\n## Test Strategy', `\n${appended}\n## Test Strategy`);
|
||||
}
|
||||
|
||||
// ---- BLOCKER 1b25ba56: appended review steps hold the 1.8 step rules on any plan
|
||||
|
||||
test('review 630 / 1b25ba56: on a 1.7 plan, an appended review step that cannot run is refused (--steps-from)', () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'r630-'));
|
||||
try {
|
||||
const plan = withTemplateStep(buildPlan({ version: '1.7', design: false }), 3);
|
||||
const p = join(dir, 'plan.md');
|
||||
writeFileSync(p, plan);
|
||||
const plain = spawnSync(process.execPath, [VALIDATOR, '--strict', '--json', p], { encoding: 'utf8' });
|
||||
assert.equal(plain.status, 0, 'the control: a 1.7 plan is not held to the 1.8 step rules');
|
||||
const r = spawnSync(process.execPath, [VALIDATOR, '--strict', '--json', '--steps-from', '3', p], { encoding: 'utf8' });
|
||||
assert.equal(r.status, 1, `the appended step is held to them: ${r.stdout}`);
|
||||
const codes = JSON.parse(r.stdout).errors.map((e) => e.code);
|
||||
assert.ok(codes.includes('PLAN_STEP_VERIFY_NOT_RUNNABLE') || codes.includes('PLAN_STEP_NO_TEST_FIRST'), codes.join(','));
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
test('review 630 / 1b25ba56: --steps-from leaves the earlier steps of a 1.7 plan alone', () => {
|
||||
const plan = buildPlan({ version: '1.7', design: false, steps: [{ testFirst: false }, { endToEnd: 'bash tests/e2e.sh' }] });
|
||||
assert.deepEqual(validatePlanContent(plan, { strict: true, stepsFrom: 3 }).errors, [],
|
||||
'step 1 has no Test first, and it is not one of the appended steps');
|
||||
});
|
||||
|
||||
test('review 630 / 1b25ba56: Phase 7.2 validates the appended steps with --steps-from', () => {
|
||||
const cmd = read('commands/trekexecute.md');
|
||||
assert.match(cmd, /plan-validator\.mjs --strict --json --steps-from \{total \+ 1\} "\{plan_path\}"/);
|
||||
});
|
||||
|
||||
// ---- 57a5e67c: the wave path names the guard's refusal ------------------------
|
||||
|
||||
test('review 630 / 57a5e67c: the wave merge path says what to do when the guard refused because no one can be asked', () => {
|
||||
const cmd = read('commands/trekexecute.md');
|
||||
const at = cmd.indexOf('**Merge session branches back (SEQUENTIAL, one at a time):**');
|
||||
const before = cmd.slice(cmd.lastIndexOf('- **`all_completed: true`:**', at), at);
|
||||
assert.match(before, /no one to ask/, 'the guard-refused case is named on the wave path');
|
||||
assert.match(before, /by hand/, 'and the way on: the operator runs the merge by hand');
|
||||
assert.match(before, /wave\.json/, 'wave.json is kept');
|
||||
});
|
||||
|
||||
// ---- 2cf29d6f / 727924c0: what counts as a step of only tests ------------------
|
||||
|
||||
test('review 630: a comma inside a Files: note is not a path', () => {
|
||||
const body = '- **Files:** `tests/a.test.mjs` (new: covers x, y)\n- **Changes:** add the test\n';
|
||||
assert.deepEqual(stepTestsOnly(body), { testsOnly: true, files: ['tests/a.test.mjs'] });
|
||||
});
|
||||
|
||||
test('review 630: a data file under specs/ is not a test', () => {
|
||||
const body = '- **Files:** `api/specs/openapi.yaml`\n- **Changes:** add the endpoint\n';
|
||||
assert.equal(stepTestsOnly(body).testsOnly, false);
|
||||
const fixture = '- **Files:** `tests/fixtures/users.json`\n- **Changes:** add the fixture\n';
|
||||
assert.equal(stepTestsOnly(fixture).testsOnly, true, 'the control: a fixture under tests/ is a test file');
|
||||
});
|
||||
|
||||
test('review 630: a tests-only End-to-end step is told where End-to-end belongs', () => {
|
||||
const plan = buildPlan({ steps: [{}, { file: 'tests/e2e.sh', test: 'tests/e2e.sh', endToEnd: 'bash tests/e2e.sh' }] });
|
||||
const w = validatePlanContent(plan, { strict: true }).warnings.find((x) => x.code === 'PLAN_STEP_TESTS_ONLY');
|
||||
assert.ok(w, 'the warning fires: red proof would reject this step');
|
||||
assert.match(`${w.message} ${w.hint ?? ''}`, /`End-to-end:` (rides|belongs) on the last step that builds/);
|
||||
assert.match(read('templates/plan-template.md'), /`End-to-end:` (rides|belongs) on the last step that builds/);
|
||||
});
|
||||
|
||||
// ---- c4623362: a removed line can still be cited --------------------------------
|
||||
|
||||
test('review 630 / c4623362: the out-of-range hint says how to cite a line that no longer exists', () => {
|
||||
const r = validateFindings({ reviewer: 'r', findings: [{
|
||||
id: 'a'.repeat(40), severity: 'MAJOR', rule_key: 'MISSING_TEST', file: 'package.json', line: 1,
|
||||
brief_ref: 'x', title: 't', recommended_action: 'a',
|
||||
detail: 'the check formerly at package.json:9999 was removed',
|
||||
}] }, { repoRoot: ROOT });
|
||||
const e = r.errors.find((x) => x.code === 'FINDING_PROSE_LINE_OUT_OF_RANGE');
|
||||
assert.ok(e, 'the control: the plain citation is still checked');
|
||||
assert.match(e.hint, /<rev>:<path>:<line>/);
|
||||
const ok = validateFindings({ reviewer: 'r', findings: [{
|
||||
id: 'a'.repeat(40), severity: 'MAJOR', rule_key: 'MISSING_TEST', file: 'package.json', line: 1,
|
||||
brief_ref: 'x', title: 't', recommended_action: 'a',
|
||||
detail: 'the check formerly at v6.2.0:package.json:9999 was removed',
|
||||
}] }, { repoRoot: ROOT });
|
||||
assert.deepEqual(ok.errors, [], 'a revision-qualified citation is not judged against today\'s file');
|
||||
assert.match(read('commands/trekreview.md'), /<rev>:<path>:<line>/, 'the re-ask names the form');
|
||||
});
|
||||
|
||||
// ---- 2261016d: a 499-line SKILL.md is under the 500-line limit -------------------
|
||||
|
||||
test('review 630 / 2261016d: a SKILL.md of 499 lines ending in a newline passes the size rule', () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'r630-skill-'));
|
||||
try {
|
||||
const skill = join(dir, 'test-first');
|
||||
cpSync(join(ROOT, 'skills', 'test-first'), skill, { recursive: true });
|
||||
const text = readFileSync(join(skill, 'SKILL.md'), 'utf8').replace(/\n+$/, '\n');
|
||||
const lines = text.split('\n').length - 1;
|
||||
assert.ok(lines < 499, `the fixture pads the real skill (${lines} lines)`);
|
||||
const padded = text.replace('\n## Credits', `${'\n'.repeat(499 - lines)}\n## Credits`);
|
||||
assert.equal(padded.split('\n').length - 1, 499);
|
||||
writeFileSync(join(skill, 'SKILL.md'), padded);
|
||||
const codes = lintSkill(skill).errors.map((e) => e.code);
|
||||
assert.ok(!codes.includes('SKILL_TOO_LONG'), codes.join(','));
|
||||
writeFileSync(join(skill, 'SKILL.md'), padded + '\n');
|
||||
assert.ok(lintSkill(skill).errors.map((e) => e.code).includes('SKILL_TOO_LONG'), 'the control: 500 lines fail');
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
// ---- cb34c32e: regression-check takes commandEnv from the shared util ------------
|
||||
|
||||
test('review 630 / cb34c32e: regression-check imports commandEnv from lib/util/command-env.mjs', () => {
|
||||
const src = read('lib/verification/regression-check.mjs');
|
||||
assert.match(src, /import \{ commandEnv \} from '\.\.\/util\/command-env\.mjs';/);
|
||||
assert.doesNotMatch(src, /from '\.\/red-proof\.mjs'/);
|
||||
assert.ok(src.indexOf('function exec(') > src.lastIndexOf('\nimport '), 'exec is defined after the imports');
|
||||
});
|
||||
|
||||
// ---- CLAUDE.md names the mode-read exemptions as they are -----------------------
|
||||
|
||||
test('review 630: CLAUDE.md counts the mode-read exemptions the launch test holds', () => {
|
||||
const src = read('tests/lib/no-claude-code-launch.test.mjs');
|
||||
const n = (src.match(/\['mode-read', \d+,/g) || []).length;
|
||||
assert.equal(n, 4, 'the launch test holds four mode-read entries');
|
||||
assert.match(read('CLAUDE.md'), /four files that only read\s+(?:> )?the session's permission mode/);
|
||||
});
|
||||
|
||||
// ---- /security-review, older than v6.2.0: a read-only git that writes a file ----
|
||||
|
||||
test('review 630 / security: `git diff --output=<path>` is not a read-only check', async () => {
|
||||
const { allowedCommand } = await import('../../lib/verification/criteria-runner.mjs');
|
||||
assert.equal(allowedCommand('git diff --stat').allowed, true, 'the control: a read-only git check runs');
|
||||
assert.equal(allowedCommand('git log --output-indicator-new=+ -1').allowed, true, 'an option that only starts with --output is not it');
|
||||
for (const cmd of ['git diff --output=../../.zshrc', 'git diff --output ../x', 'git log -p --output=x', 'git show HEAD --output=x']) {
|
||||
const r = allowedCommand(cmd);
|
||||
assert.equal(r.allowed, false, `${cmd} writes a file`);
|
||||
assert.match(r.reason, /--output/);
|
||||
}
|
||||
});
|
||||
|
|
@ -85,6 +85,7 @@ const DOORS = `### Doors
|
|||
function fixture({
|
||||
suiteExit = 0, doors = DOORS, verdict = 'ALLOW', writeReview = true,
|
||||
reviewSha = 'work', afterRelease = null, dirty = false, version = '1.0.1',
|
||||
testScript = null, files = {}, attributes = null,
|
||||
} = {}) {
|
||||
const base = mkdtempSync(join(tmpdir(), 'release-check-'));
|
||||
const repo = join(base, 'repo');
|
||||
|
|
@ -94,12 +95,14 @@ function fixture({
|
|||
git(repo, 'config', 'user.name', 't');
|
||||
git(repo, 'config', 'commit.gpgsign', 'false');
|
||||
|
||||
const pkg = (v) => JSON.stringify({ name: 'fx', version: v, scripts: { test: `node -e "process.exit(${suiteExit})"` } }, null, 2) + '\n';
|
||||
const pkg = (v) => JSON.stringify({ name: 'fx', version: v, scripts: { test: testScript ?? `node -e "process.exit(${suiteExit})"` } }, null, 2) + '\n';
|
||||
writeFileSync(join(repo, 'package.json'), pkg('1.0.0'));
|
||||
writeFileSync(join(repo, '.claude-plugin', 'plugin.json'), JSON.stringify({ name: 'fx', version: '1.0.0' }) + '\n');
|
||||
writeFileSync(join(repo, 'README.md'), '# fx\n\n\n\nText.\n');
|
||||
writeFileSync(join(repo, 'CHANGELOG.md'), '# Changelog\n\n## v1.0.0 — 2026-10-01 — first\n\n- first\n');
|
||||
writeFileSync(join(repo, 'a.mjs'), 'export const a = 1;\n');
|
||||
for (const [rel, text] of Object.entries(files)) writeFileSync(join(repo, rel), text);
|
||||
if (attributes) writeFileSync(join(repo, '.git', 'info', 'attributes'), attributes);
|
||||
git(repo, 'add', '-A');
|
||||
git(repo, 'commit', '-q', '-m', 'feat: first');
|
||||
writeFileSync(join(repo, 'a.mjs'), 'export const a = 2;\n');
|
||||
|
|
@ -301,3 +304,59 @@ test('release-check — releaseSection matches the CHANGELOG heading dialects ex
|
|||
assert.match(mod.releaseSection(text, '1.10.0'), /ten/);
|
||||
assert.equal(mod.releaseSection(text, '2.0.0'), null);
|
||||
});
|
||||
|
||||
// ---- release review of 6.3.0 (2026-10-10): what counts as bookkeeping ----------
|
||||
|
||||
const AFTER_PKG = JSON.stringify({ name: 'fx', version: '1.0.1', scripts: { test: 'node -e "process.exit(0)" && echo changed' } }, null, 2) + '\n';
|
||||
|
||||
test('release-check — a change to package.json after the review other than its version → 1', () => {
|
||||
const fx = fixture({ afterRelease: { 'package.json': AFTER_PKG } });
|
||||
try {
|
||||
const r = run(fx);
|
||||
assert.equal(r.status, 1, r.stdout + r.stderr);
|
||||
assert.match(r.stdout, /package\.json \(lines other than the version\)/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
||||
test('release-check — a change to plugin.json after the review other than its version → 1', () => {
|
||||
const fx = fixture({ afterRelease: { '.claude-plugin/plugin.json': JSON.stringify({ name: 'fx', version: '1.0.1', hooks: './x.json' }) + '\n' } });
|
||||
try {
|
||||
const r = run(fx);
|
||||
assert.equal(r.status, 1, r.stdout + r.stderr);
|
||||
assert.match(r.stdout, /plugin\.json \(lines other than the version\)/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
||||
test('release-check — a README line that starts with "++ " is a change, not a diff header → 1', () => {
|
||||
const fx = fixture({ afterRelease: { 'README.md': '# fx\n\n\n\nText.\n++ new claim\n' } });
|
||||
try {
|
||||
const r = run(fx);
|
||||
assert.equal(r.status, 1, r.stdout + r.stderr);
|
||||
assert.match(r.stdout, /README\.md \(lines other than the version badge\)/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
||||
test('release-check — a README diff that cannot be read → 2 (never green)', () => {
|
||||
const fx = fixture({ attributes: 'README.md diff=broken\n' });
|
||||
try {
|
||||
spawnSync('git', ['config', 'diff.broken.textconv', 'false'], { cwd: fx.repo });
|
||||
const r = run(fx);
|
||||
assert.equal(r.status, 2, r.stdout + r.stderr);
|
||||
assert.match(r.stdout, /README\.md/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
||||
test('release-check — a red suite is red when the check itself runs under node --test', () => {
|
||||
const fx = fixture({
|
||||
testScript: 'node --test red.test.mjs',
|
||||
files: { 'red.test.mjs': "import { test } from 'node:test';\nimport assert from 'node:assert';\ntest('red', () => assert.equal(1, 2));\n" },
|
||||
});
|
||||
try {
|
||||
const control = run(fx);
|
||||
assert.equal(control.status, 1, `the control: the suite is red: ${control.stdout}`);
|
||||
const r = spawnSync(process.execPath, [CHECK, '--repo', fx.repo, '--review', fx.review],
|
||||
{ encoding: 'utf8', env: { ...process.env, NODE_TEST_CONTEXT: 'child-v8' } });
|
||||
assert.equal(r.status, 1, `under a parent test runner: ${r.stdout}`);
|
||||
assert.match(r.stdout, /npm test exited/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue