fix: the findings of the release review of 6.3.0
The review of v6.2.0..f669b87 (BLOCK: 1 BLOCKER, 12 MAJOR, 1 MINOR), each fixed behind a test that failed first, or, for the two missing tests, a test shown to fail on a stub of the branch it covers. - Review steps appended by /trekexecute Phase 7.2 are held to the step rules on any plan version: plan-validator --steps-from <N> (on a plan below 1.8 a template review step validated and was executed). - The guard refuses where no one can be asked also when its rules will not load; the wave merge path names that refusal instead of MERGE_REFUSED. Tests for the classifier-error refusal and post-approval-log's own check. - release-check: the version files may change only in their version after the review (JSON compared without it), README diff lines are read inside hunks only, an unreadable diff is exit 2, and npm test runs without NODE_TEST_CONTEXT. - What counts as a step of only tests: a note in parentheses is not a path, and a data file under specs/ is not a test. End-to-end: rides on the last step that builds behaviour (validator hint, trekplan, template). - A removed line is cited as <rev>:<path>:<line> (findings-schema hint, trekreview re-ask). - skill-lint counts a trailing newline as no line; regression-check takes commandEnv from lib/util/command-env.mjs; CLAUDE.md names four mode-read exemptions. - /security-review, older than v6.2.0: the criteria runner refuses git --output, which writes a file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
f669b8775f
commit
bef9c5b3ac
SSH key fingerprint:
SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
18 changed files with 450 additions and 43 deletions
|
|
@ -21,9 +21,10 @@ Voyage — a contract-driven Claude Code pipeline: brief, research, plan, execut
|
|||
> `tests/lib/no-claude-code-launch.test.mjs`; its exemption list holds historical
|
||||
> mentions plus one operator instrument (operator decision, 2026-10-02:
|
||||
> `docs/proevesett.md` tells a person how to run Anthropic's plugin eval tool on
|
||||
> the fixed test set; Voyage itself starts nothing) plus two files that only read
|
||||
> the session's permission mode from hook input (operator decision, 2026-10-06),
|
||||
> each with a reason and an exact count.
|
||||
> the fixed test set; Voyage itself starts nothing) plus four files that only read
|
||||
> the session's permission mode from hook input, two hooks and their tests
|
||||
> (operator decisions, 2026-10-06 and 2026-10-09), each with a reason and an
|
||||
> exact count.
|
||||
|
||||
## Commands
|
||||
|
||||
|
|
|
|||
|
|
@ -915,7 +915,13 @@ in this order:
|
|||
a merge when the branch is main), so that answer is the one question for the
|
||||
merge (operator decision 2026-10-08; it was two when `--gates` added a pause
|
||||
of its own). A declined guard question means the call did not run: stop and
|
||||
report it like `MERGE_REFUSED`, keeping `wave.json`.
|
||||
report it like `MERGE_REFUSED`, keeping `wave.json`. In the permission mode
|
||||
that shows no prompts the guard does not ask: it refuses the push and the
|
||||
merge (exit 2, "no one to ask"). Then stop, keep `wave.json`, and report that
|
||||
the guard refused because no one can be asked, not that local changes are in
|
||||
the way: the operator runs this session's push-and-merge block by hand, or
|
||||
switches to a mode that asks and re-runs. A re-run in the same mode is
|
||||
refused again.
|
||||
|
||||
**Merge session branches back (SEQUENTIAL, one at a time):**
|
||||
|
||||
|
|
@ -2216,9 +2222,12 @@ the run is `stopped`, and the merge stays closed.
|
|||
plan_version ≥ 1.7 plan the manifest the legacy rule synthesizes
|
||||
(Phase 2: `expected_paths` ← its Files, …): a rule already in this file, not
|
||||
a new design. Earlier steps are never touched.
|
||||
3. Check that the new steps can run, before anyone is asked:
|
||||
3. Check that the new steps can run, before anyone is asked. `--steps-from`
|
||||
holds the appended steps to the step rules (`Test first:` with its `File:`,
|
||||
one runnable `Verify:`) whatever the plan's version; below 1.8 nothing else
|
||||
would:
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/plan-validator.mjs --strict --json "{plan_path}"
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/plan-validator.mjs --strict --json --steps-from {total + 1} "{plan_path}"
|
||||
```
|
||||
Any exit but 0 (a `Verify:` that is not one test-runner call, a
|
||||
`Test first:` without its `File:`, …): restore the plan from the copy,
|
||||
|
|
|
|||
|
|
@ -755,7 +755,9 @@ Write the plan following the template structure. The plan MUST include:
|
|||
`End-to-end: \`bash tests/e2e.sh\``. The same command is also listed under
|
||||
`## Verification`: Phase 7 of `/trekexecute` runs that section, and a passed
|
||||
criterion that runs the step's `End-to-end:` command is what verifies the
|
||||
run end to end.
|
||||
run end to end. `End-to-end:` rides on the last step that builds behaviour,
|
||||
with `tests/e2e.sh` in that step's `Files:`: a step of its own that only adds
|
||||
the script is a step of only tests, and it can never be proven red.
|
||||
10. **Execution Strategy** — For plans with > 5 steps: group steps into sessions
|
||||
(3–5 steps each), organize sessions into waves (parallel where independent),
|
||||
specify scope fences per session. Omit for plans with ≤ 5 steps.
|
||||
|
|
|
|||
|
|
@ -423,7 +423,9 @@ same stable error codes (`FINDINGS_NO_JSON_BLOCK`, `FINDINGS_PARSE_ERROR`,
|
|||
`FINDING_PROSE_LINE_OUT_OF_RANGE` is the same check on a `path:line` written
|
||||
in prose. In a finding's own fields (`title`, `detail`, `recommended_action`,
|
||||
`brief_ref`) it is an **error**: re-ask that reviewer like any other schema
|
||||
failure. In the prose around the JSON block it is a **warning**: the re-ask
|
||||
failure, and quote the way a removed line is cited, `<rev>:<path>:<line>`
|
||||
(`v6.2.0:hooks/x.mjs:120`, not judged against today's file), so a finding
|
||||
about deleted code keeps its line instead of losing it. In the prose around the JSON block it is a **warning**: the re-ask
|
||||
asks for the block only and cannot correct that prose, so do not re-ask —
|
||||
report each such warning (reviewer, citation, the file's line count) in the
|
||||
Coverage section of `review.md`.
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ import { readFileSync } from 'node:fs';
|
|||
// The guard's own modules are loaded so that a broken install cannot crash
|
||||
// the hook: a crash (exit 1) is a non-blocking error to Claude Code, which
|
||||
// would let every command through. A module that will not load makes the
|
||||
// guard ask about every command instead.
|
||||
// guard ask about every command instead, and refuse it where no one can be asked.
|
||||
let lib;
|
||||
let loadError = null;
|
||||
try {
|
||||
|
|
@ -217,7 +217,22 @@ if (!command || typeof command !== 'string') {
|
|||
process.exit(0);
|
||||
}
|
||||
|
||||
// In the permission mode where Claude Code shows no prompts, an "ask" reaches
|
||||
// no one and the call runs unattended (see the refusal below).
|
||||
const NO_PROMPT_MODE = 'bypassPermissions';
|
||||
const noOneToAsk = input.permission_mode === NO_PROMPT_MODE;
|
||||
|
||||
if (loadError) {
|
||||
// A guard that cannot read its rules asks about every command; where no one
|
||||
// can be asked it refuses instead (release review of 6.3.0, 2026-10-10).
|
||||
if (noOneToAsk) {
|
||||
process.stderr.write(
|
||||
`[voyage] BLOCKED: Voyage's Bash guard could not load its rules (${loadError.message}) and there is no one to ask\n` +
|
||||
' This session runs in the permission mode where Claude Code asks nothing, so the guard cannot ask a person. ' +
|
||||
'Way forward: reinstall Voyage to restore the guard, or run the command yourself.\n'
|
||||
);
|
||||
process.exit(2);
|
||||
}
|
||||
process.stdout.write(JSON.stringify({
|
||||
hookSpecificOutput: {
|
||||
hookEventName: 'PreToolUse',
|
||||
|
|
@ -280,8 +295,7 @@ try {
|
|||
// decisions 2026-10-09: first merge and push, then the other two classes).
|
||||
// /trekexecute relies on this: its main-merge gate is a lock, and this guard's
|
||||
// question is the one question.
|
||||
const NO_PROMPT_MODE = 'bypassPermissions';
|
||||
const unattended = input.permission_mode === NO_PROMPT_MODE ? approvals : [];
|
||||
const unattended = noOneToAsk ? approvals : [];
|
||||
if (unattended.length > 0) {
|
||||
process.stderr.write(
|
||||
`[voyage] BLOCKED: ${unattended.map((m) => m.name).join(', ')} with no one to ask\n` +
|
||||
|
|
|
|||
|
|
@ -288,10 +288,12 @@ export function onFailureAction(text) {
|
|||
* The paths a field names. Each entry (comma- or line-separated, a sub-bullet's
|
||||
* `-` dropped) gives its backticked spans, or else its first word: a plain
|
||||
* path beside a backticked one is still a path (independent review, order 10).
|
||||
* A parenthesised note is not an entry: "`a.test.mjs` (new: covers x, y)"
|
||||
* names one path, not "y)" too (release review of 6.3.0, 2026-10-10).
|
||||
*/
|
||||
function fieldPaths(text) {
|
||||
const out = [];
|
||||
for (const raw of String(text ?? '').split(/[,\n]/)) {
|
||||
for (const raw of String(text ?? '').replace(/\([^()\n]*\)/g, ' ').split(/[,\n]/)) {
|
||||
const entry = raw.trim().replace(/^[-*]\s+/, '');
|
||||
if (entry === '') continue;
|
||||
const spans = [...entry.matchAll(/`([^`\n]+)`/g)].map((m) => m[1].trim());
|
||||
|
|
@ -303,7 +305,11 @@ function fieldPaths(text) {
|
|||
|
||||
// A document is never the test here: `docs/specs/api.md` sits under a `specs/`
|
||||
// folder, which lib/util/test-path.mjs reads as tests for the floor guard.
|
||||
// Nor is a data file under a `spec/` or `specs/` folder (`api/specs/openapi.yaml`
|
||||
// is an API spec); under tests/ it is a fixture, and a fixture is a test file.
|
||||
const DOC_RE = /\.(md|markdown|mdx|txt|rst|adoc)$/i;
|
||||
const DATA_RE = /\.(ya?ml|json|toml|xml|csv)$/i;
|
||||
const UNDER_TESTS_RE = /(^|\/)(tests?|__tests__)\//;
|
||||
|
||||
/**
|
||||
* Whether every file in a step's `Files:` is a test: the step's own `Test
|
||||
|
|
@ -324,7 +330,8 @@ export function stepTestsOnly(body) {
|
|||
const testFirst = f.has('Test first')
|
||||
? [...f.get('Test first').text.matchAll(/\bFile:\**[ \t]*`([^`\n]+)`/gi)].map((m) => m[1].trim())
|
||||
: [];
|
||||
const isTest = (p) => testFirst.includes(p) || (isTestPath(p) && !DOC_RE.test(p));
|
||||
const isTest = (p) => testFirst.includes(p) ||
|
||||
(isTestPath(p) && !DOC_RE.test(p) && !(DATA_RE.test(p) && !UNDER_TESTS_RE.test(p)));
|
||||
return { testsOnly: files.length > 0 && files.every(isTest), files };
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -162,7 +162,9 @@ function checkProseCitations(text, where, repoRoot, errors, seen) {
|
|||
errors.push(issue(
|
||||
'FINDING_PROSE_LINE_OUT_OF_RANGE',
|
||||
`${where} cites ${m[0]}, but ${m[1]} has ${n} lines`,
|
||||
'Cite the line number in the file itself (Read it), not the line in the diff or patch, or drop the line number.',
|
||||
'Cite the line number in the file itself (Read it), not the line in the diff or patch. A line that is no ' +
|
||||
'longer in the file is cited with its revision, `<rev>:<path>:<line>` (`v6.2.0:hooks/x.mjs:120`), which ' +
|
||||
'is not judged against today\'s file.',
|
||||
));
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -145,7 +145,8 @@ export function lintSkill(skillDir, opts = {}) {
|
|||
if (/lint-exempt/i.test(text)) errors.push(err('SKILL_SELF_EXEMPTION', 'a skill may not exempt itself; exemptions live in EXEMPTIONS in lib/skills/skill-lint.mjs'));
|
||||
|
||||
// Size
|
||||
const lineCount = text.split('\n').length;
|
||||
// A file that ends in a newline has no line after it (findings-schema counts the same way).
|
||||
const lineCount = text.endsWith('\n') ? text.split('\n').length - 1 : text.split('\n').length;
|
||||
if (lineCount >= MAX_LINES) errors.push(err('SKILL_TOO_LONG', `SKILL.md has ${lineCount} lines (must stay under ${MAX_LINES}); move detail to a linked file`));
|
||||
|
||||
// Body: headings, enforcement, references, shell injection
|
||||
|
|
|
|||
|
|
@ -164,13 +164,19 @@ function checkRecovery(sections, opts) {
|
|||
`Open with one of ${ON_FAILURE_ACTIONS.join(' | ')}, then the specifics (\`retry once, then escalate\`).`));
|
||||
}
|
||||
if (stepTestsOnly(s.body).testsOnly) {
|
||||
// The run-the-thing script alone is such a step too: `End-to-end:` rides on
|
||||
// the last step that builds behaviour (release review of 6.3.0, 2026-10-10).
|
||||
const e2e = f.has('End-to-end')
|
||||
? ' `End-to-end:` rides on the last step that builds behaviour, with its script in that step\'s `Files:`, ' +
|
||||
'not on a step of its own.'
|
||||
: '';
|
||||
warnings.push(issue('PLAN_STEP_TESTS_ONLY',
|
||||
`Step ${s.n}: every file in \`Files:\` is a test; a step of only tests can never be proven red`,
|
||||
'The test for a behaviour belongs in the step that builds it: at this step\'s start point the behaviour ' +
|
||||
'already exists (an earlier step built it, or it was there before the plan), so its test passes there and ' +
|
||||
'red proof rejects it (`not-red-at-base`). Move the test into the step that builds the behaviour; a test ' +
|
||||
'meant to pass from the start (a golden or characterization test) is the `Regression:` of the step that ' +
|
||||
'changes the code, not a step of its own.'));
|
||||
'changes the code, not a step of its own.' + e2e));
|
||||
}
|
||||
}
|
||||
return { errors, warnings };
|
||||
|
|
@ -286,6 +292,27 @@ export function validatePlanContent(rawText, opts = {}) {
|
|||
else warnings.push(...recovery.errors);
|
||||
warnings.push(...recovery.warnings);
|
||||
|
||||
// Steps a run appended (the review loop's remediation steps, /trekexecute
|
||||
// Phase 7.2) must be runnable steps whatever the plan's version: below 1.8
|
||||
// the step rules never ran, and a review step whose Verify is "re-run
|
||||
// /trekreview" validated and was executed (release review of 6.3.0,
|
||||
// 2026-10-10). From 1.8 the rules below already cover every step.
|
||||
const stepsFrom = Number.isInteger(opts.stepsFrom) && opts.stepsFrom > 0 ? opts.stepsFrom : null;
|
||||
if (stepsFrom !== null && !versionAtLeast(planVersion, '1.8')) {
|
||||
const appended = sections.filter((s) => Number(s.n) >= stepsFrom);
|
||||
const stepIssues = [
|
||||
...checkSteps(appended).filter((i) => i.code !== 'PLAN_NO_END_TO_END'),
|
||||
...checkRunnable('', appended, { cwd: opts.cwd }),
|
||||
];
|
||||
for (const s of appended) {
|
||||
if (!parseStepFields(s.body).has('Verify')) {
|
||||
stepIssues.push(issue('PLAN_STEP_VERIFY_NOT_RUNNABLE', `Step ${s.n}: no \`Verify:\` field`,
|
||||
'An appended step needs one test command: proof of red and the done line run it.'));
|
||||
}
|
||||
}
|
||||
errors.push(...stepIssues);
|
||||
}
|
||||
|
||||
const ceremony = extractPlanCeremony(text);
|
||||
if (versionAtLeast(planVersion, '1.8')) {
|
||||
const v18 = [...checkSteps(sections), ...checkRunnable(text, sections, { cwd: opts.cwd })];
|
||||
|
|
@ -352,12 +379,14 @@ function isMain() {
|
|||
if (isMain()) {
|
||||
const args = process.argv.slice(2);
|
||||
const strict = !args.includes('--soft');
|
||||
const filePath = args.find(a => !a.startsWith('--'));
|
||||
if (!filePath) {
|
||||
process.stderr.write('Usage: plan-validator.mjs [--strict|--soft] <plan.md>\n');
|
||||
const at = args.indexOf('--steps-from');
|
||||
const stepsFrom = at >= 0 ? Number(args[at + 1]) : null;
|
||||
const filePath = args.find((a, i) => !a.startsWith('--') && !(at >= 0 && i === at + 1));
|
||||
if (!filePath || (at >= 0 && !(Number.isInteger(stepsFrom) && stepsFrom > 0))) {
|
||||
process.stderr.write('Usage: plan-validator.mjs [--strict|--soft] [--json] [--steps-from <N>] <plan.md>\n');
|
||||
process.exit(2);
|
||||
}
|
||||
const r = validatePlan(filePath, { strict });
|
||||
const r = validatePlan(filePath, { strict, stepsFrom });
|
||||
if (args.includes('--json')) {
|
||||
process.stdout.write(JSON.stringify({
|
||||
valid: r.valid,
|
||||
|
|
|
|||
|
|
@ -315,10 +315,13 @@ const RUNNERS = {
|
|||
bash: (t) => (isTestScript(t[1]) ? permit() : deny('bash, but not a script under tests/')),
|
||||
// The system bash by its path (macOS: 3.2), so a check can pin the shell.
|
||||
'/bin/bash': (t) => (isTestScript(t[1]) ? permit() : deny('/bin/bash, but not a script under tests/')),
|
||||
git: (t) =>
|
||||
READONLY_GIT.has(t[1])
|
||||
? permit()
|
||||
: deny(`git, but not a read-only subcommand first (${[...READONLY_GIT].join('|')})`),
|
||||
git: (t) => {
|
||||
if (!READONLY_GIT.has(t[1])) return deny(`git, but not a read-only subcommand first (${[...READONLY_GIT].join('|')})`);
|
||||
// `--output` makes diff/log/show write their output to a file (release
|
||||
// review of 6.3.0, 2026-10-10: a plan's check could overwrite any file).
|
||||
if (t.some((w) => w === '--output' || w.startsWith('--output='))) return deny('git with --output writes a file, so it is not a read-only check');
|
||||
return permit();
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -24,23 +24,23 @@
|
|||
// progress.json uses (C15). Nothing else is translated.
|
||||
//
|
||||
// The command runs the way the runner runs it (`/bin/sh -c`), but without
|
||||
// NODE_TEST_CONTEXT in its environment (red-proof.mjs `commandEnv`): measured
|
||||
// NODE_TEST_CONTEXT in its environment (lib/util/command-env.mjs): measured
|
||||
// 2026-10-08, a red `node --test` regression read as passed when this CLI was
|
||||
// itself started under `node --test`, because the nested run reported to the
|
||||
// parent and exited 0.
|
||||
|
||||
function exec(command, { cwd, timeoutMs }) {
|
||||
const r = spawnSync('/bin/sh', ['-c', command], { cwd, encoding: 'utf8', timeout: timeoutMs, env: commandEnv() });
|
||||
return { status: r.status, stdout: r.stdout ?? '', stderr: r.stderr ?? '' };
|
||||
}
|
||||
|
||||
import { readFileSync, realpathSync } from 'node:fs';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { sliceSteps, parseStepFields } from '../parsers/plan-schema.mjs';
|
||||
import { firstCommand, runCriteria } from './criteria-runner.mjs';
|
||||
import { commandEnv } from './red-proof.mjs';
|
||||
import { commandEnv } from '../util/command-env.mjs';
|
||||
|
||||
function exec(command, { cwd, timeoutMs }) {
|
||||
const r = spawnSync('/bin/sh', ['-c', command], { cwd, encoding: 'utf8', timeout: timeoutMs, env: commandEnv() });
|
||||
return { status: r.status, stdout: r.stdout ?? '', stderr: r.stderr ?? '' };
|
||||
}
|
||||
|
||||
export const REGRESSION_FIELD = 'Regression';
|
||||
const EXIT = Object.freeze({ passed: 0, absent: 0, failed: 1, blocked: 1, unrunnable: 1, error: 2 });
|
||||
|
|
|
|||
|
|
@ -19,7 +19,8 @@
|
|||
// stops (1), undecided stops (2)
|
||||
// coverage the review is of THIS history: its `scope_sha_end` is HEAD or an
|
||||
// ancestor of HEAD, and the commits after it touch only the release
|
||||
// bookkeeping (version files, CHANGELOG, the README version badge)
|
||||
// bookkeeping (the CHANGELOG, the version in the version files,
|
||||
// the README version badge)
|
||||
// door the CHANGELOG section for the version has a `### Doors` list with
|
||||
// one `- one-way: …` or `- two-way: …` line per delivery
|
||||
//
|
||||
|
|
@ -37,6 +38,7 @@ import { spawnSync } from 'node:child_process';
|
|||
import { dirname, join, relative, resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { decideVerdict } from '../lib/review/verdict-gate.mjs';
|
||||
import { commandEnv } from '../lib/util/command-env.mjs';
|
||||
import { parseDocument } from '../lib/util/frontmatter.mjs';
|
||||
|
||||
const PLUGIN_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
|
||||
|
|
@ -44,12 +46,27 @@ const PLUGIN_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
|
|||
export const EXIT = Object.freeze({ READY: 0, NOT_READY: 1, UNDECIDED: 2 });
|
||||
export const CHECK_IDS = Object.freeze(['tree', 'suite', 'review', 'coverage', 'door']);
|
||||
|
||||
// Files a release commit may change after the reviewed commit. README.md is
|
||||
// allowed only for lines that carry the version badge.
|
||||
// Files a release commit may change after the reviewed commit, and how much of
|
||||
// each (release review of 6.3.0, 2026-10-10: a changed test script or hook
|
||||
// wiring passed as bookkeeping): the CHANGELOG freely; the version files only
|
||||
// in their version (the JSON with it taken out must be equal); README.md only
|
||||
// on the lines that carry the version badge.
|
||||
export const RELEASE_BOOKKEEPING = Object.freeze([
|
||||
'CHANGELOG.md', 'package.json', 'package-lock.json', '.claude-plugin/plugin.json', 'README.md',
|
||||
]);
|
||||
const BADGE_RE = /badge\/version-/;
|
||||
const VERSION_FILES = Object.freeze(['package.json', 'package-lock.json', '.claude-plugin/plugin.json']);
|
||||
|
||||
// The JSON of a version file with its version taken out: the top-level
|
||||
// `version`, and in a lockfile the root package's `packages[""].version`.
|
||||
export function withoutVersion(text) {
|
||||
const o = JSON.parse(text);
|
||||
if (o && typeof o === 'object') {
|
||||
delete o.version;
|
||||
if (o.packages && o.packages[''] && typeof o.packages[''] === 'object') delete o.packages[''].version;
|
||||
}
|
||||
return JSON.stringify(o);
|
||||
}
|
||||
const DOOR_LINE_RE = /^- (one-way|two-way): \S/;
|
||||
|
||||
const git = (cwd, args) => spawnSync('git', args, { cwd, encoding: 'utf8' });
|
||||
|
|
@ -100,6 +117,19 @@ export function parseDoors(section) {
|
|||
return { found: true, entries, bad };
|
||||
}
|
||||
|
||||
// The +/- lines of a unified diff, read inside its hunks only: a content line
|
||||
// that starts with "++ " or "-- " looks like a file header but is not one.
|
||||
export function changedLines(diff) {
|
||||
const out = [];
|
||||
let inHunk = false;
|
||||
for (const l of diff.split('\n')) {
|
||||
if (l.startsWith('diff --git ')) inHunk = false;
|
||||
else if (l.startsWith('@@')) inHunk = true;
|
||||
else if (inHunk && /^[+-]/.test(l)) out.push(l);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function checkTree(repo) {
|
||||
const r = git(repo, ['status', '--porcelain']);
|
||||
if (r.status !== 0) return { id: 'tree', status: 'error', detail: `git status failed: ${r.stderr.trim()}` };
|
||||
|
|
@ -112,7 +142,8 @@ function checkTree(repo) {
|
|||
|
||||
function checkSuite(repo, treeClean) {
|
||||
if (!treeClean) return { id: 'suite', status: 'fail', detail: 'suite not run: the tree has uncommitted changes, so it would not measure HEAD' };
|
||||
const r = spawnSync('npm', ['test'], { cwd: repo, encoding: 'utf8', maxBuffer: 256 * 1024 * 1024 });
|
||||
// commandEnv: under a parent `node --test` the suite would report to it and exit 0 whatever it found.
|
||||
const r = spawnSync('npm', ['test'], { cwd: repo, encoding: 'utf8', maxBuffer: 256 * 1024 * 1024, env: commandEnv() });
|
||||
if (r.error) return { id: 'suite', status: 'error', detail: `npm test could not start: ${r.error.message} — not measured` };
|
||||
const out = `${r.stdout ?? ''}\n${r.stderr ?? ''}`;
|
||||
const summary = out.split('\n').filter((l) => /^ℹ (tests|pass|fail|skipped) /.test(l)).map((l) => l.slice(2));
|
||||
|
|
@ -150,10 +181,20 @@ function checkCoverage(repo, reviewText) {
|
|||
if (names.status !== 0) return { id: 'coverage', status: 'error', detail: `git diff failed: ${names.stderr.trim()}` };
|
||||
const changed = names.stdout.split('\n').filter(Boolean);
|
||||
const outside = changed.filter((f) => !RELEASE_BOOKKEEPING.includes(f));
|
||||
for (const file of changed.filter((f) => VERSION_FILES.includes(f))) {
|
||||
const before = git(repo, ['show', `${full}:${file}`]);
|
||||
const after = git(repo, ['show', `HEAD:${file}`]);
|
||||
if (before.status !== 0 || after.status !== 0) { outside.push(`${file} (added or removed after the review)`); continue; }
|
||||
let same;
|
||||
try { same = withoutVersion(before.stdout) === withoutVersion(after.stdout); } catch (e) {
|
||||
return { id: 'coverage', status: 'error', detail: `${file} is not JSON (${e.message}) — not measured` };
|
||||
}
|
||||
if (!same) outside.push(`${file} (lines other than the version)`);
|
||||
}
|
||||
if (changed.includes('README.md')) {
|
||||
const d = git(repo, ['diff', '-U0', full, 'HEAD', '--', 'README.md']);
|
||||
const moved = d.stdout.split('\n').filter((l) => /^[+-]/.test(l) && !/^(\+\+\+|---)( |$)/.test(l));
|
||||
if (moved.some((l) => !BADGE_RE.test(l))) outside.push('README.md (lines other than the version badge)');
|
||||
if (d.status !== 0) return { id: 'coverage', status: 'error', detail: `git diff of README.md failed: ${d.stderr.trim()} — not measured` };
|
||||
if (changedLines(d.stdout).some((l) => !BADGE_RE.test(l))) outside.push('README.md (lines other than the version badge)');
|
||||
}
|
||||
if (outside.length > 0) {
|
||||
return { id: 'coverage', status: 'fail', detail: `changed after the reviewed commit ${full.slice(0, 12)} and not covered by the review: ${outside.join(', ')}`, items: outside };
|
||||
|
|
|
|||
|
|
@ -75,7 +75,8 @@ own. Preparatory refactoring comes first, as its own steps. Every step has
|
|||
end to end (`End-to-end:`). The test for a behaviour belongs in the step that
|
||||
builds it: a step of only tests for behaviour an earlier step delivered passes
|
||||
at its start point and can never be proven red (the validator warns:
|
||||
`PLAN_STEP_TESTS_ONLY`).
|
||||
`PLAN_STEP_TESTS_ONLY`). So `End-to-end:` rides on the last step that builds
|
||||
behaviour, with its script in that step's `Files:`, not on a step of its own.
|
||||
|
||||
Optional per-step fields: `Interface in:` (what the step consumes, and which
|
||||
earlier step or existing code produces it), `Interface out:` (what it produces
|
||||
|
|
|
|||
|
|
@ -265,7 +265,7 @@ test('P11: a review step in the template form is caught by the plan validator, s
|
|||
assert.ok(errors.some((c) => /VERIFY_NOT_RUNNABLE|TEST_FIRST/.test(c)), errors.join(','));
|
||||
|
||||
const p = section(CMD, '## Phase 7.2 — Simplify, review, verdict (P11)');
|
||||
const check = p.indexOf('plan-validator.mjs --strict --json "{plan_path}"');
|
||||
const check = p.indexOf('plan-validator.mjs --strict --json --steps-from {total + 1} "{plan_path}"');
|
||||
const ask = p.indexOf('Approve the plan as it is now?');
|
||||
assert.ok(check > 0 && ask > check, 'the validator runs before the approval question');
|
||||
assert.match(p, /restore the plan from the copy/, 'a stop leaves the plan matching its approval');
|
||||
|
|
@ -332,6 +332,56 @@ test('P12: the guard asks before a merge into main; in the mode with no prompts
|
|||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
// Release review of 6.3.0 (2026-10-10): the refusal covers the guard's two
|
||||
// fallbacks too — rules that will not load, and a command it could not read.
|
||||
function guardIn(plugin, cwd, command, permission_mode) {
|
||||
return node([join(plugin, 'hooks', 'scripts', 'pre-bash-executor.mjs')], {
|
||||
input: JSON.stringify({ tool_name: 'Bash', tool_input: { command }, cwd, permission_mode, session_id: 'coupling', tool_use_id: `t${Math.random()}` }),
|
||||
});
|
||||
}
|
||||
|
||||
test('P12 / review 630: a guard whose rules will not load refuses where no one is asked', async () => {
|
||||
const { pluginCopy } = await import('../hooks/guard-helpers.mjs');
|
||||
const plugin = pluginCopy({ 'hooks/lib/git-approval.mjs': 'missing' });
|
||||
const dir = repo();
|
||||
try {
|
||||
const asked = guardIn(plugin, dir, 'git status', 'default');
|
||||
assert.equal(asked.status, 0, asked.stderr);
|
||||
assert.match(asked.stdout, /"permissionDecision":"ask"/, 'the control: where someone can answer, the broken guard asks');
|
||||
const r = guardIn(plugin, dir, 'git push origin main', NO_PROMPT_MODE);
|
||||
assert.equal(r.status, 2, `refused, not asked: ${r.stdout}${r.stderr}`);
|
||||
assert.match(r.stderr, /no one to ask/);
|
||||
assert.match(r.stderr, /could not load/);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
rmSync(plugin, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('P12 / review 630: a command the classifier could not read is asked, and refused where no one is asked', async () => {
|
||||
const { pluginCopy } = await import('../hooks/guard-helpers.mjs');
|
||||
const plugin = pluginCopy();
|
||||
const lib = join(plugin, 'hooks', 'lib', 'git-approval.mjs');
|
||||
const src = readFileSync(lib, 'utf8');
|
||||
assert.ok(src.includes('export function classifyGitApproval('), 'the copy carries the classifier');
|
||||
writeFileSync(lib, src.replace('export function classifyGitApproval(',
|
||||
'export function classifyGitApproval() { throw new Error(\'classifier broke\'); }\nfunction classifyGitApprovalUnused('));
|
||||
const dir = repo();
|
||||
try {
|
||||
const asked = guardIn(plugin, dir, 'git push origin feature', 'default');
|
||||
assert.equal(asked.status, 0, asked.stderr);
|
||||
assert.match(asked.stdout, /"permissionDecision":"ask"/, 'where someone can answer it is asked');
|
||||
assert.match(asked.stdout, /could not read/);
|
||||
const r = guardIn(plugin, dir, 'git push origin feature', NO_PROMPT_MODE);
|
||||
assert.equal(r.status, 2, `refused: ${r.stdout}${r.stderr}`);
|
||||
assert.match(r.stderr, /could not read/);
|
||||
assert.match(r.stderr, /no one to ask/);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
rmSync(plugin, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// ---- the other new progress fields (Handover 5, additive-optional) -----------
|
||||
|
||||
test('H5: the wave-2 fields are shape-checked when present and optional when absent', () => {
|
||||
|
|
|
|||
2
tests/fixtures/jsonl-schemas.md
vendored
2
tests/fixtures/jsonl-schemas.md
vendored
|
|
@ -22,7 +22,7 @@
|
|||
| trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason | commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) | trekbrief.md:1130-1145 | profile, phase_models, profile_source | none |
|
||||
| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | lib/stats/stats-append.mjs (trekresearch.md Stats tracking) | trekresearch.md:782-808 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
|
||||
| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | lib/stats/stats-append.mjs (trekplan.md Phase 12) | trekplan.md:1115-1135 | profile, phase_models, parallel_agents, profile_source | none |
|
||||
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:2858-2880 | profile, phase_models, profile_source | none |
|
||||
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:2867-2889 | profile, phase_models, profile_source | none |
|
||||
| trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:78-99 | payload.profile, payload.phase_models, payload.profile_source | none |
|
||||
| trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) |
|
||||
| trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | lib/stats/stats-append.mjs (trekreview.md Phase 8) | trekreview.md:628-632 | profile, phase_models, profile_source | none |
|
||||
|
|
|
|||
|
|
@ -109,11 +109,12 @@ test('review 8: a command nobody was asked about is not logged, even if it reads
|
|||
// Operator decision 2026-10-09: where no person can be asked, the guard refuses
|
||||
// a merge into main or a push (exit 2) instead of asking, so the call never runs
|
||||
// and nothing reaches the approval log. post-approval-log's own check of the
|
||||
// recorded mode stays as a second line.
|
||||
// recorded mode stays as a second line (its own test below).
|
||||
const NO_PROMPT_MODE = 'bypassPermissions';
|
||||
test('uncertain (no-prompt mode): the push to main is refused, and no approval is logged', async () => {
|
||||
const plugin = pluginCopy({ 'lib/util/approval-log.mjs': 'stub' });
|
||||
const repo = makeRepo('main');
|
||||
const base = { session_id: `s-b-${RUN}`, tool_use_id: 'toolu_b', cwd: repo, tool_name: 'Bash', tool_input: { command: 'git push origin main' }, permission_mode: 'bypassPermissions' };
|
||||
const base = { session_id: `s-b-${RUN}`, tool_use_id: 'toolu_b', cwd: repo, tool_name: 'Bash', tool_input: { command: 'git push origin main' }, permission_mode: NO_PROMPT_MODE };
|
||||
const pre = await runHook(join(plugin, 'hooks', 'scripts', 'pre-bash-executor.mjs'), { ...base, hook_event_name: 'PreToolUse' });
|
||||
assert.equal(pre.code, 2, `refused, not asked: ${JSON.stringify(pre)}`);
|
||||
const log = join(tmp(), 'stub.jsonl');
|
||||
|
|
@ -121,6 +122,25 @@ test('uncertain (no-prompt mode): the push to main is refused, and no approval i
|
|||
assert.equal(readStubLog(log).length, 0);
|
||||
});
|
||||
|
||||
// The second line on its own (release review of 6.3.0, 2026-10-10): a record
|
||||
// that says no person was asked is never logged, whoever wrote it.
|
||||
test('uncertain (no-prompt mode): a recorded question no one saw is not logged', async () => {
|
||||
const plugin = pluginCopy({ 'lib/util/approval-log.mjs': 'stub' });
|
||||
const repo = makeRepo('main');
|
||||
const { writeApprovalRecord } = await import(join(plugin, 'hooks', 'lib', 'approval-record.mjs'));
|
||||
const ids = { session_id: `s-r-${RUN}`, tool_use_id: 'toolu_r' };
|
||||
assert.ok(writeApprovalRecord(ids.session_id, ids.tool_use_id, {
|
||||
command: 'git push origin main', permission_mode: NO_PROMPT_MODE,
|
||||
matches: [{ kind: 'merge', name: 'push to main', subject: repo }], asked_at: new Date().toISOString(),
|
||||
}), 'the record is written');
|
||||
const log = join(tmp(), 'stub.jsonl');
|
||||
const r = await runHookWithEnv(join(plugin, 'hooks', 'scripts', 'post-approval-log.mjs'), {
|
||||
...ids, hook_event_name: 'PostToolUse', cwd: repo, tool_name: 'Bash', tool_input: { command: 'git push origin main' },
|
||||
}, { STUB_LOG: log });
|
||||
assert.match(parseJsonOut(r.stdout).hookSpecificOutput.additionalContext, /not logged/i);
|
||||
assert.equal(readStubLog(log).length, 0);
|
||||
});
|
||||
|
||||
// 9. Classification
|
||||
const kinds = (cmd, branch = 'feat', cwd) => classifyGitApproval(cmd, { branch, cwd }).map((m) => m.kind);
|
||||
for (const cmd of ['git tag -v v1', 'git tag --verify v1', 'git push -n', 'git push --dry-run origin main']) {
|
||||
|
|
|
|||
166
tests/lib/review-630.test.mjs
Normal file
166
tests/lib/review-630.test.mjs
Normal file
|
|
@ -0,0 +1,166 @@
|
|||
// tests/lib/review-630.test.mjs
|
||||
// The findings of the release review of 6.3.0 (2026-10-10), each held by a
|
||||
// test that failed on f669b87. The guard's and the release check's own tests
|
||||
// live next to their files (tests/commands/trekexecute-coupling.test.mjs,
|
||||
// tests/hooks/guards-review-regressions.test.mjs, tests/scripts/release-check.test.mjs).
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { cpSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { buildPlan } from '../helpers/plan-v18-builder.mjs';
|
||||
import { validatePlanContent } from '../../lib/validators/plan-validator.mjs';
|
||||
import { stepTestsOnly } from '../../lib/parsers/plan-schema.mjs';
|
||||
import { validateFindings } from '../../lib/review/findings-schema.mjs';
|
||||
import { lintSkill } from '../../lib/skills/skill-lint.mjs';
|
||||
|
||||
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
||||
const read = (rel) => readFileSync(join(ROOT, rel), 'utf8');
|
||||
const VALIDATOR = join(ROOT, 'lib', 'validators', 'plan-validator.mjs');
|
||||
|
||||
// The review template's own Step R1, appended the Phase 7.2 way.
|
||||
function withTemplateStep(plan, n) {
|
||||
const r1 = /### Step R1:[^\n]*\n[\s\S]*?(?=\n> )/.exec(read('templates/trekreview-template.md'));
|
||||
assert.ok(r1, 'templates/trekreview-template.md carries a ### Step R1');
|
||||
// Phase 7.2 step 2 also adds the manifest the legacy rule synthesizes (expected_paths ← Files).
|
||||
const appended = r1[0].replace(/^### Step R1:/, `### Step ${n}:`) +
|
||||
'- **Checkpoint:** `git commit -m "fix(review): x [01234567]"`\n' +
|
||||
'- **Manifest:**\n ```yaml\n manifest:\n expected_paths:\n - lib/foo.mjs\n min_file_count: 1\n' +
|
||||
' commit_message_pattern: "^fix\\\\(review\\\\):"\n bash_syntax_check: []\n forbidden_paths: []\n must_contain: []\n ```\n';
|
||||
return plan.replace('\n## Test Strategy', `\n${appended}\n## Test Strategy`);
|
||||
}
|
||||
|
||||
// ---- BLOCKER 1b25ba56: appended review steps hold the 1.8 step rules on any plan
|
||||
|
||||
test('review 630 / 1b25ba56: on a 1.7 plan, an appended review step that cannot run is refused (--steps-from)', () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'r630-'));
|
||||
try {
|
||||
const plan = withTemplateStep(buildPlan({ version: '1.7', design: false }), 3);
|
||||
const p = join(dir, 'plan.md');
|
||||
writeFileSync(p, plan);
|
||||
const plain = spawnSync(process.execPath, [VALIDATOR, '--strict', '--json', p], { encoding: 'utf8' });
|
||||
assert.equal(plain.status, 0, 'the control: a 1.7 plan is not held to the 1.8 step rules');
|
||||
const r = spawnSync(process.execPath, [VALIDATOR, '--strict', '--json', '--steps-from', '3', p], { encoding: 'utf8' });
|
||||
assert.equal(r.status, 1, `the appended step is held to them: ${r.stdout}`);
|
||||
const codes = JSON.parse(r.stdout).errors.map((e) => e.code);
|
||||
assert.ok(codes.includes('PLAN_STEP_VERIFY_NOT_RUNNABLE') || codes.includes('PLAN_STEP_NO_TEST_FIRST'), codes.join(','));
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
test('review 630 / 1b25ba56: --steps-from leaves the earlier steps of a 1.7 plan alone', () => {
|
||||
const plan = buildPlan({ version: '1.7', design: false, steps: [{ testFirst: false }, { endToEnd: 'bash tests/e2e.sh' }] });
|
||||
assert.deepEqual(validatePlanContent(plan, { strict: true, stepsFrom: 3 }).errors, [],
|
||||
'step 1 has no Test first, and it is not one of the appended steps');
|
||||
});
|
||||
|
||||
test('review 630 / 1b25ba56: Phase 7.2 validates the appended steps with --steps-from', () => {
|
||||
const cmd = read('commands/trekexecute.md');
|
||||
assert.match(cmd, /plan-validator\.mjs --strict --json --steps-from \{total \+ 1\} "\{plan_path\}"/);
|
||||
});
|
||||
|
||||
// ---- 57a5e67c: the wave path names the guard's refusal ------------------------
|
||||
|
||||
test('review 630 / 57a5e67c: the wave merge path says what to do when the guard refused because no one can be asked', () => {
|
||||
const cmd = read('commands/trekexecute.md');
|
||||
const at = cmd.indexOf('**Merge session branches back (SEQUENTIAL, one at a time):**');
|
||||
const before = cmd.slice(cmd.lastIndexOf('- **`all_completed: true`:**', at), at);
|
||||
assert.match(before, /no one to ask/, 'the guard-refused case is named on the wave path');
|
||||
assert.match(before, /by hand/, 'and the way on: the operator runs the merge by hand');
|
||||
assert.match(before, /wave\.json/, 'wave.json is kept');
|
||||
});
|
||||
|
||||
// ---- 2cf29d6f / 727924c0: what counts as a step of only tests ------------------
|
||||
|
||||
test('review 630: a comma inside a Files: note is not a path', () => {
|
||||
const body = '- **Files:** `tests/a.test.mjs` (new: covers x, y)\n- **Changes:** add the test\n';
|
||||
assert.deepEqual(stepTestsOnly(body), { testsOnly: true, files: ['tests/a.test.mjs'] });
|
||||
});
|
||||
|
||||
test('review 630: a data file under specs/ is not a test', () => {
|
||||
const body = '- **Files:** `api/specs/openapi.yaml`\n- **Changes:** add the endpoint\n';
|
||||
assert.equal(stepTestsOnly(body).testsOnly, false);
|
||||
const fixture = '- **Files:** `tests/fixtures/users.json`\n- **Changes:** add the fixture\n';
|
||||
assert.equal(stepTestsOnly(fixture).testsOnly, true, 'the control: a fixture under tests/ is a test file');
|
||||
});
|
||||
|
||||
test('review 630: a tests-only End-to-end step is told where End-to-end belongs', () => {
|
||||
const plan = buildPlan({ steps: [{}, { file: 'tests/e2e.sh', test: 'tests/e2e.sh', endToEnd: 'bash tests/e2e.sh' }] });
|
||||
const w = validatePlanContent(plan, { strict: true }).warnings.find((x) => x.code === 'PLAN_STEP_TESTS_ONLY');
|
||||
assert.ok(w, 'the warning fires: red proof would reject this step');
|
||||
assert.match(`${w.message} ${w.hint ?? ''}`, /`End-to-end:` (rides|belongs) on the last step that builds/);
|
||||
assert.match(read('templates/plan-template.md'), /`End-to-end:` (rides|belongs) on the last step that builds/);
|
||||
});
|
||||
|
||||
// ---- c4623362: a removed line can still be cited --------------------------------
|
||||
|
||||
test('review 630 / c4623362: the out-of-range hint says how to cite a line that no longer exists', () => {
|
||||
const r = validateFindings({ reviewer: 'r', findings: [{
|
||||
id: 'a'.repeat(40), severity: 'MAJOR', rule_key: 'MISSING_TEST', file: 'package.json', line: 1,
|
||||
brief_ref: 'x', title: 't', recommended_action: 'a',
|
||||
detail: 'the check formerly at package.json:9999 was removed',
|
||||
}] }, { repoRoot: ROOT });
|
||||
const e = r.errors.find((x) => x.code === 'FINDING_PROSE_LINE_OUT_OF_RANGE');
|
||||
assert.ok(e, 'the control: the plain citation is still checked');
|
||||
assert.match(e.hint, /<rev>:<path>:<line>/);
|
||||
const ok = validateFindings({ reviewer: 'r', findings: [{
|
||||
id: 'a'.repeat(40), severity: 'MAJOR', rule_key: 'MISSING_TEST', file: 'package.json', line: 1,
|
||||
brief_ref: 'x', title: 't', recommended_action: 'a',
|
||||
detail: 'the check formerly at v6.2.0:package.json:9999 was removed',
|
||||
}] }, { repoRoot: ROOT });
|
||||
assert.deepEqual(ok.errors, [], 'a revision-qualified citation is not judged against today\'s file');
|
||||
assert.match(read('commands/trekreview.md'), /<rev>:<path>:<line>/, 'the re-ask names the form');
|
||||
});
|
||||
|
||||
// ---- 2261016d: a 499-line SKILL.md is under the 500-line limit -------------------
|
||||
|
||||
test('review 630 / 2261016d: a SKILL.md of 499 lines ending in a newline passes the size rule', () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'r630-skill-'));
|
||||
try {
|
||||
const skill = join(dir, 'test-first');
|
||||
cpSync(join(ROOT, 'skills', 'test-first'), skill, { recursive: true });
|
||||
const text = readFileSync(join(skill, 'SKILL.md'), 'utf8').replace(/\n+$/, '\n');
|
||||
const lines = text.split('\n').length - 1;
|
||||
assert.ok(lines < 499, `the fixture pads the real skill (${lines} lines)`);
|
||||
const padded = text.replace('\n## Credits', `${'\n'.repeat(499 - lines)}\n## Credits`);
|
||||
assert.equal(padded.split('\n').length - 1, 499);
|
||||
writeFileSync(join(skill, 'SKILL.md'), padded);
|
||||
const codes = lintSkill(skill).errors.map((e) => e.code);
|
||||
assert.ok(!codes.includes('SKILL_TOO_LONG'), codes.join(','));
|
||||
writeFileSync(join(skill, 'SKILL.md'), padded + '\n');
|
||||
assert.ok(lintSkill(skill).errors.map((e) => e.code).includes('SKILL_TOO_LONG'), 'the control: 500 lines fail');
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
// ---- cb34c32e: regression-check takes commandEnv from the shared util ------------
|
||||
|
||||
test('review 630 / cb34c32e: regression-check imports commandEnv from lib/util/command-env.mjs', () => {
|
||||
const src = read('lib/verification/regression-check.mjs');
|
||||
assert.match(src, /import \{ commandEnv \} from '\.\.\/util\/command-env\.mjs';/);
|
||||
assert.doesNotMatch(src, /from '\.\/red-proof\.mjs'/);
|
||||
assert.ok(src.indexOf('function exec(') > src.lastIndexOf('\nimport '), 'exec is defined after the imports');
|
||||
});
|
||||
|
||||
// ---- CLAUDE.md names the mode-read exemptions as they are -----------------------
|
||||
|
||||
test('review 630: CLAUDE.md counts the mode-read exemptions the launch test holds', () => {
|
||||
const src = read('tests/lib/no-claude-code-launch.test.mjs');
|
||||
const n = (src.match(/\['mode-read', \d+,/g) || []).length;
|
||||
assert.equal(n, 4, 'the launch test holds four mode-read entries');
|
||||
assert.match(read('CLAUDE.md'), /four files that only read\s+(?:> )?the session's permission mode/);
|
||||
});
|
||||
|
||||
// ---- /security-review, older than v6.2.0: a read-only git that writes a file ----
|
||||
|
||||
test('review 630 / security: `git diff --output=<path>` is not a read-only check', async () => {
|
||||
const { allowedCommand } = await import('../../lib/verification/criteria-runner.mjs');
|
||||
assert.equal(allowedCommand('git diff --stat').allowed, true, 'the control: a read-only git check runs');
|
||||
assert.equal(allowedCommand('git log --output-indicator-new=+ -1').allowed, true, 'an option that only starts with --output is not it');
|
||||
for (const cmd of ['git diff --output=../../.zshrc', 'git diff --output ../x', 'git log -p --output=x', 'git show HEAD --output=x']) {
|
||||
const r = allowedCommand(cmd);
|
||||
assert.equal(r.allowed, false, `${cmd} writes a file`);
|
||||
assert.match(r.reason, /--output/);
|
||||
}
|
||||
});
|
||||
|
|
@ -85,6 +85,7 @@ const DOORS = `### Doors
|
|||
function fixture({
|
||||
suiteExit = 0, doors = DOORS, verdict = 'ALLOW', writeReview = true,
|
||||
reviewSha = 'work', afterRelease = null, dirty = false, version = '1.0.1',
|
||||
testScript = null, files = {}, attributes = null,
|
||||
} = {}) {
|
||||
const base = mkdtempSync(join(tmpdir(), 'release-check-'));
|
||||
const repo = join(base, 'repo');
|
||||
|
|
@ -94,12 +95,14 @@ function fixture({
|
|||
git(repo, 'config', 'user.name', 't');
|
||||
git(repo, 'config', 'commit.gpgsign', 'false');
|
||||
|
||||
const pkg = (v) => JSON.stringify({ name: 'fx', version: v, scripts: { test: `node -e "process.exit(${suiteExit})"` } }, null, 2) + '\n';
|
||||
const pkg = (v) => JSON.stringify({ name: 'fx', version: v, scripts: { test: testScript ?? `node -e "process.exit(${suiteExit})"` } }, null, 2) + '\n';
|
||||
writeFileSync(join(repo, 'package.json'), pkg('1.0.0'));
|
||||
writeFileSync(join(repo, '.claude-plugin', 'plugin.json'), JSON.stringify({ name: 'fx', version: '1.0.0' }) + '\n');
|
||||
writeFileSync(join(repo, 'README.md'), '# fx\n\n\n\nText.\n');
|
||||
writeFileSync(join(repo, 'CHANGELOG.md'), '# Changelog\n\n## v1.0.0 — 2026-10-01 — first\n\n- first\n');
|
||||
writeFileSync(join(repo, 'a.mjs'), 'export const a = 1;\n');
|
||||
for (const [rel, text] of Object.entries(files)) writeFileSync(join(repo, rel), text);
|
||||
if (attributes) writeFileSync(join(repo, '.git', 'info', 'attributes'), attributes);
|
||||
git(repo, 'add', '-A');
|
||||
git(repo, 'commit', '-q', '-m', 'feat: first');
|
||||
writeFileSync(join(repo, 'a.mjs'), 'export const a = 2;\n');
|
||||
|
|
@ -301,3 +304,59 @@ test('release-check — releaseSection matches the CHANGELOG heading dialects ex
|
|||
assert.match(mod.releaseSection(text, '1.10.0'), /ten/);
|
||||
assert.equal(mod.releaseSection(text, '2.0.0'), null);
|
||||
});
|
||||
|
||||
// ---- release review of 6.3.0 (2026-10-10): what counts as bookkeeping ----------
|
||||
|
||||
const AFTER_PKG = JSON.stringify({ name: 'fx', version: '1.0.1', scripts: { test: 'node -e "process.exit(0)" && echo changed' } }, null, 2) + '\n';
|
||||
|
||||
test('release-check — a change to package.json after the review other than its version → 1', () => {
|
||||
const fx = fixture({ afterRelease: { 'package.json': AFTER_PKG } });
|
||||
try {
|
||||
const r = run(fx);
|
||||
assert.equal(r.status, 1, r.stdout + r.stderr);
|
||||
assert.match(r.stdout, /package\.json \(lines other than the version\)/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
||||
test('release-check — a change to plugin.json after the review other than its version → 1', () => {
|
||||
const fx = fixture({ afterRelease: { '.claude-plugin/plugin.json': JSON.stringify({ name: 'fx', version: '1.0.1', hooks: './x.json' }) + '\n' } });
|
||||
try {
|
||||
const r = run(fx);
|
||||
assert.equal(r.status, 1, r.stdout + r.stderr);
|
||||
assert.match(r.stdout, /plugin\.json \(lines other than the version\)/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
||||
test('release-check — a README line that starts with "++ " is a change, not a diff header → 1', () => {
|
||||
const fx = fixture({ afterRelease: { 'README.md': '# fx\n\n\n\nText.\n++ new claim\n' } });
|
||||
try {
|
||||
const r = run(fx);
|
||||
assert.equal(r.status, 1, r.stdout + r.stderr);
|
||||
assert.match(r.stdout, /README\.md \(lines other than the version badge\)/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
||||
test('release-check — a README diff that cannot be read → 2 (never green)', () => {
|
||||
const fx = fixture({ attributes: 'README.md diff=broken\n' });
|
||||
try {
|
||||
spawnSync('git', ['config', 'diff.broken.textconv', 'false'], { cwd: fx.repo });
|
||||
const r = run(fx);
|
||||
assert.equal(r.status, 2, r.stdout + r.stderr);
|
||||
assert.match(r.stdout, /README\.md/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
||||
test('release-check — a red suite is red when the check itself runs under node --test', () => {
|
||||
const fx = fixture({
|
||||
testScript: 'node --test red.test.mjs',
|
||||
files: { 'red.test.mjs': "import { test } from 'node:test';\nimport assert from 'node:assert';\ntest('red', () => assert.equal(1, 2));\n" },
|
||||
});
|
||||
try {
|
||||
const control = run(fx);
|
||||
assert.equal(control.status, 1, `the control: the suite is red: ${control.stdout}`);
|
||||
const r = spawnSync(process.execPath, [CHECK, '--repo', fx.repo, '--review', fx.review],
|
||||
{ encoding: 'utf8', env: { ...process.env, NODE_TEST_CONTEXT: 'child-v8' } });
|
||||
assert.equal(r.status, 1, `under a parent test runner: ${r.stdout}`);
|
||||
assert.match(r.stdout, /npm test exited/);
|
||||
} finally { fx.cleanup(); }
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue