open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

fix: round 7 of the release review of 6.3.0, for 6.3.1

Browse source 
Round 7 (516dffa..45aa9e1, WARN: 12 MAJOR, 0 BLOCKER) and the fence defect
found while writing its review.md. Each fix began with a test that failed on
0f400d3 (v6.3.0); suite 2208 (2206/0/2), 14 tests more.

- Tests-only (operator 2026-10-11): the plan's Files: alone decides, and the
  text says so (red-proof, trekexecute Sub-step B, step-implementer,
  HANDOVER-CONTRACTS). red-proof no longer reads the step's diff, so a change
  already in the tree when the step began no longer counts against it, and a
  plan placed outside .claude/ reads the same. The round-6 test that had the
  diff overrule the plan is rewritten to the new rule.
- The Files: reader keeps a plain file beside a backticked path
  (", Dockerfile", "the CLI entry src/cli.mjs", "and Makefile"). Measured on
  the 520 Files: fields in 182 plans: no field reads differently.
- Red proof reports a command its timeout killed as killed after the timeout,
  not "could not start"; the release check tells a timeout from output past
  its buffer and from another signal. One reading of a stop
  (criteria-runner stoppedBy) for red proof, the criteria runner and the
  release check.
- A trailing `# comment` is dropped in a backticked check too, and in a
  command passed as written, so the words screened are the words run.
- The release check reads a relative --review in --repo.
- The 15-minute timeout is defined once (criteria-runner.mjs).
- The findings check reads each cited file once per validation.
- One isMain helper (lib/util/is-main.mjs) for every CLI: the 31 copies of
  the realpath guard and the 20 modules still on the file:// guard, which
  exited 0 having done nothing under a symlink or a path with a space.
- The Bash guard writes its no-one-to-ask refusal once, on a direct condition.
- The findings check ends a ```json block only at a fence on its own line, so
  three backticks inside a JSON string no longer end it (also in 6.2.0).
- Brief-conformance notes: a test for test-lock green on exit 127 (it
  passed at once; it fails when 127 is read as could-not-run); the prose now
  names the git revision `@~1` as the one leading-@ exception.

Not changed: the example plan's steps 1-2 keep their tests-only warning
(operator decision 2026-10-09; the r5 test records it). Brief note 4 (a
Regression: whose runner is not installed goes to the fix rounds) is a
routing question for the operator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-11 09:11:57 +02:00
commit 29b4ac91b3
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
65 changed files with 582 additions and 485 deletions
Download patch file Download diff file Expand all files Collapse all files

13
tests/hooks/test-lock-green.test.mjs
View file

@ -94,3 +94,16 @@ test('trekexecute: D1 marks the lock green through the command, a fix round mark
assert.match(md, /test-lock\.mjs lock --files \{test files\} --step \{N\} --cmd "\{Verify command\}"/);
assert.match(md, /test-lock\.mjs pending/);
});
test('green: exit 127 from the test program itself is a red -> exit 1, no mark (release review of 6.3.0, round 7)', () => {
const repo = makeRepo('main');
mkdirSync(join(repo, 'tests'), { recursive: true });
// The tool the step builds is not there yet: bash runs, and exits 127 for it.
writeFileSync(join(repo, 'tests', 'e2e.sh'), './bin/tool\n');
const cmd = 'bash tests/e2e.sh';
assert.equal(cli(repo, ['lock', '--files', 'tests/e2e.sh', '--step', '3', '--cmd', cmd]).code, 0);
const g = cli(repo, ['green', '--cmd', cmd]);
assert.equal(g.code, 1, g.stderr);
assert.match(g.stderr, /exited 127/);
assert.equal(JSON.parse(cli(repo, ['status']).stdout).green_at, null);
});

15
tests/lib/doc-consistency.test.mjs
View file

@ -209,11 +209,12 @@ test('HANDOVER-CONTRACTS.md contains Handover 7 section (session-state)', () =>
});
test('review-validator has CLI shim', () => {
// Ordre 5: the realpath form (no-op-free under a symlink or a space); the run
// through a symlink is proven in tests/validators/review-validator.test.mjs.
// Ordre 5: the realpath form (no-op-free under a symlink or a space), shared
// since 6.3.1 (lib/util/is-main.mjs); the run through a symlink is proven in
// tests/validators/review-validator.test.mjs.
const text = read('lib/validators/review-validator.mjs');
assert.ok(
text.includes('realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url))'),
text.includes('if (isMain(import.meta.url))'),
'lib/validators/review-validator.mjs should expose the realpath CLI shim so commands can call it from Bash',
);
});
@ -221,18 +222,18 @@ test('review-validator has CLI shim', () => {
test('session-state-validator has CLI shim', () => {
const text = read('lib/validators/session-state-validator.mjs');
assert.ok(
text.includes('import.meta.url === '),
text.includes('if (isMain(import.meta.url))'),
'lib/validators/session-state-validator.mjs should expose the standard CLI shim ' +
'(if (import.meta.url === `file://${process.argv[1]}`)) so /trekcontinue can call it from Bash',
'(if (isMain(import.meta.url))) so /trekcontinue can call it from Bash',
);
});
test('next-session-prompt-validator has CLI shim', () => {
const text = read('lib/validators/next-session-prompt-validator.mjs');
assert.ok(
text.includes('import.meta.url === '),
text.includes('if (isMain(import.meta.url))'),
'lib/validators/next-session-prompt-validator.mjs should expose the standard CLI shim ' +
'(if (import.meta.url === `file://${process.argv[1]}`)) so /trekcontinue Phase 1.5 can call it from Bash',
'(if (isMain(import.meta.url))) so /trekcontinue Phase 1.5 can call it from Bash',
);
});

184
tests/lib/review-630.test.mjs
View file

@ -4,10 +4,12 @@
// live next to their files (tests/commands/trekexecute-coupling.test.mjs,
// tests/hooks/guards-review-regressions.test.mjs, tests/scripts/release-check.test.mjs).
import { test } from 'node:test';
import { test, mock } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import fs from 'node:fs';
import { cpSync, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';
import { syncBuiltinESMExports } from 'node:module';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
@ -640,7 +642,10 @@ test('review 630 r6 / plan-schema.mjs:318: a backticked file name with no extens
assert.deepEqual(stepTestsOnly(step('`tests/a.test.mjs` — covers `parse` and `JSON`')).files, ['tests/a.test.mjs'], 'a span inside a description is no path');
});
test('review 630 r6 / plan-schema.mjs:329 (altitude): red proof checks tests-only against the step\'s own diff', async () => {
// Round 6 had red proof overrule the plan with the step's diff; the operator
// reversed that on 2026-10-11 (round 7): the plan's Files: alone decides, and a
// plain Makefile beside a backticked test is now read from Files: itself.
test('review 630 r6 / plan-schema.mjs:329 (reversed in r7): a Makefile beside a backticked test is read from Files:, not from the diff', async () => {
const { proveRed } = await import('../../lib/verification/red-proof.mjs');
const work = mkdtempSync(join(tmpdir(), 'r630-diff-'));
const repo = gitRepo('r630-diff-repo-');
@ -650,16 +655,14 @@ test('review 630 r6 / plan-schema.mjs:329 (altitude): red proof checks tests-onl
mkdirSync(join(repo, 'tests'));
writeFileSync(join(repo, 'tests', 'a.test.mjs'), "import { test } from 'node:test';\ntest('a', () => {});\n");
writeFileSync(join(repo, 'Makefile'), 'test:\n\tnode --test tests/\n');
// The prose reads only the test (a plain Makefile beside backticked paths is a description word).
const plan = join(work, 'plan.md');
writeFileSync(plan, '## Implementation Plan\n\n### Step 1: x\n\n- **Files:** `tests/a.test.mjs`, Makefile\n- **Changes:** x\n');
const o = { cwd: repo, base, cmd: 'node --test tests/a.test.mjs', tests: ['tests/a.test.mjs'], plan, step: 1 };
const r = proveRed(o);
assert.equal(r.reason, 'not-red-at-base', JSON.stringify(r));
assert.equal(r.tests_only, false, `the step's diff changes Makefile: ${JSON.stringify(r.tests_only_note ?? null)}`);
assert.match(r.tests_only_note ?? '', /Makefile/);
assert.equal(r.tests_only, false, 'Files: names Makefile');
rmSync(join(repo, 'Makefile'));
assert.equal(proveRed(o).tests_only, true, 'the control: a step whose diff holds only its test is tests-only');
assert.equal(proveRed(o).tests_only, false, 'the diff is not read: without Makefile in the tree the plan still names it');
} finally { rmSync(work, { recursive: true, force: true }); rmSync(repo, { recursive: true, force: true }); }
});
@ -704,3 +707,170 @@ test('review 630 r6 / hooks are ASCII: the lines this release added to pre-bash-
assert.ok(block.includes('Everything this guard would ask about'));
assert.doesNotMatch(block, /[^\x00-\x7f]/, 'a hook script is ASCII (~/.claude/CLAUDE.md, Miljø)');
});
// ==== round 7 of the release review (2026-10-11), fixed in 6.3.1; each red on 0f400d3 ====
test('review 630 r7 / 48939bcb d20c5619 (operator 2026-10-11): the plan\'s Files: alone decides tests-only, whatever else is in the tree', async () => {
const { proveRed } = await import('../../lib/verification/red-proof.mjs');
const repo = gitRepo('r630-r7-files-');
try {
writeFileSync(join(repo, 'README.md'), 'x\n');
const base = commitAll(repo, 'base');
// Work the plan does not know about, there before the step began (trekexecute
// Sub-step 0 lets a step start on a dirty tree and logs it as a decision).
writeFileSync(join(repo, 'README.md'), 'edited before the step\n');
writeFileSync(join(repo, 'notes.txt'), 'untracked, not ignored\n');
// A plan placed outside .claude/, with the executor's files beside it (brief note 2).
mkdirSync(join(repo, 'plans', 'steps'), { recursive: true });
const plan = join(repo, 'plans', 'plan.md');
writeFileSync(plan, '## Implementation Plan\n\n### Step 1: x\n\n- **Files:** `tests/a.test.mjs`\n- **Changes:** x\n\n' +
'### Step 2: y\n\n- **Files:** `tests/a.test.mjs`, `src/a.mjs`\n- **Changes:** y\n');
writeFileSync(join(repo, 'plans', 'progress.json'), '{}\n');
writeFileSync(join(repo, 'plans', 'steps', 'step-1.report.md'), 'report\n');
// The helper wrote the step's test alone (step-implementer, mode build, step 1); it passes at the start point.
mkdirSync(join(repo, 'tests'));
writeFileSync(join(repo, 'tests', 'a.test.mjs'), "import { test } from 'node:test';\ntest('a', () => {});\n");
const o = { cwd: repo, base, cmd: 'node --test tests/a.test.mjs', tests: ['tests/a.test.mjs'], plan };
for (const redOnly of [true, false]) {
const one = proveRed({ ...o, step: 1, redOnly });
assert.equal(one.reason, 'not-red-at-base', JSON.stringify(one));
assert.equal(one.tests_only, true, `step 1's Files: names only its test (redOnly ${redOnly}): ${JSON.stringify(one.tests_only_note ?? null)}`);
const two = proveRed({ ...o, step: 2, redOnly });
assert.equal(two.reason, 'not-red-at-base', JSON.stringify(two));
assert.equal(two.tests_only, false, `step 2's Files: names src/a.mjs, though the tree holds only its test (redOnly ${redOnly})`);
}
} finally { rmSync(repo, { recursive: true, force: true }); }
});
test('review 630 r7 / 48939bcb (operator 2026-10-11): the text says plainly that the plan\'s Files: alone decides', () => {
for (const rel of ['commands/trekexecute.md', 'agents/step-implementer.md', 'lib/verification/red-proof.mjs', 'docs/HANDOVER-CONTRACTS.md']) {
const text = read(rel);
assert.match(text, /`Files:` alone decides/, `${rel} says that the plan's Files: alone decides`);
assert.doesNotMatch(text, /every file (?:the step|it) changed since (?:its|the) start point/, `${rel} no longer reads the step's diff for tests-only`);
}
const row = read('commands/trekexecute.md').split('\n').find((l) => l.startsWith('| 1, `not-red-at-base`, `tests_only: true`'));
assert.match(row, /`Files:` alone decides/, 'the routing row itself says it');
});
test('review 630 r7 / 0180a979: a file named without backticks beside a backticked path is read', () => {
const step = (files) => `- **Files:** ${files}\n- **Changes:** x\n`;
for (const [files, paths] of [
['`tests/x.test.mjs`, Dockerfile', ['tests/x.test.mjs', 'Dockerfile']],
['`tests/x.test.mjs`, the CLI entry src/cli.mjs', ['tests/x.test.mjs', 'src/cli.mjs']],
['`tests/x.test.mjs` and Makefile', ['tests/x.test.mjs', 'Makefile']],
['`tests/x.test.mjs` and `Makefile`', ['tests/x.test.mjs', 'Makefile']],
]) {
assert.deepEqual(stepTestsOnly(step(files)), { testsOnly: false, files: paths }, files);
}
// The control: a description beside a backticked path is still no path.
for (const files of ['`tests/a.test.mjs` — covers `parse` and `JSON`', '`tests/a.test.mjs` — covers pass/fail, JSON, YAML',
'`tests/a.test.mjs` (v6.3.0 format)', '`tests/a.test.mjs`: Covers the Parser, e.g. its Profile']) {
assert.deepEqual(stepTestsOnly(step(files)), { testsOnly: true, files: ['tests/a.test.mjs'] }, files);
}
});
test('review 630 r7 / 633546914: a command its timeout killed is reported as killed by the timeout, not as "could not start"', async () => {
const { proveRed } = await import('../../lib/verification/red-proof.mjs');
const repo = gitRepo('r630-r7-hang-');
try {
writeFileSync(join(repo, 'README.md'), 'x\n');
const base = commitAll(repo, 'base');
mkdirSync(join(repo, 'tests'));
writeFileSync(join(repo, 'tests', 'hang.test.mjs'), "import { test } from 'node:test';\ntest('hangs', () => new Promise((r) => setTimeout(r, 6000)));\n");
const r = proveRed({ cwd: repo, base, cmd: 'node --test tests/hang.test.mjs', tests: ['tests/hang.test.mjs'], redOnly: true, timeoutMs: 2000 });
assert.equal(r.status, 'not-run', JSON.stringify(r));
assert.doesNotMatch(r.reason, /could not start/, r.reason);
assert.match(r.reason, /killed .*after the 2 s timeout/, r.reason);
} finally { rmSync(repo, { recursive: true, force: true }); }
});
test('review 630 r7 / 5b2063e2: a trailing # comment is no part of a backticked check either, and never reaches the runner', async () => {
const { firstCommand, allowedCommand, runCommand } = await import('../../lib/verification/criteria-runner.mjs');
assert.equal(firstCommand('`npm test # full suite`').command, 'npm test');
assert.equal(firstCommand('`node --test tests/a.test.mjs #1` — expected: pass').command, 'node --test tests/a.test.mjs');
const plan = '## Implementation Plan\n\n### Step 1: x\n\n- **Files:** `src/a.mjs`\n- **Changes:** x\n' +
'- **Regression:** `npm test # full suite`\n';
const r = validatePlanContent(plan, { strict: false });
const codes = [...r.errors, ...r.warnings].map((i) => i.code);
assert.ok(!codes.includes('PLAN_STEP_REGRESSION_NOT_RUNNABLE'), JSON.stringify(codes));
// The executor passes the Verify span to red proof, the done line and the test lock as written.
assert.equal(allowedCommand('npm test # full suite', { cwd: ROOT }).allowed, true);
assert.equal(runCommand('node -e process.exit(process.argv.length-1) # two words', { cwd: ROOT }).status, 0, 'the comment\'s words reach no program');
assert.equal(firstCommand('`node --test tests/a#b.test.mjs`').command, 'node --test tests/a#b.test.mjs', 'the control: a # inside a word is no comment');
});
test('review 630 r7 / aa08f47f: the 15-minute timeout is defined once', async () => {
const files = spawnSync('git', ['ls-files', 'lib', 'scripts', 'hooks'], { cwd: ROOT, encoding: 'utf8' }).stdout.split('\n').filter((p) => p.endsWith('.mjs'));
assert.ok(files.length > 50, `listed ${files.length} files`);
assert.deepEqual(files.filter((p) => /\b900_?000\b/.test(read(p))), ['lib/verification/criteria-runner.mjs']);
const rp = await import('../../lib/verification/red-proof.mjs');
const cr = await import('../../lib/verification/criteria-runner.mjs');
assert.equal(rp.DEFAULT_TIMEOUT_MS, cr.DEFAULT_TIMEOUT_MS);
});
test('review 630 r7 / efa7de84: a file cited many times is read once per validation', () => {
const cite = 'commands/trekexecute.md:1';
const findings = Array.from({ length: 40 }, (_, i) => ({
severity: 'MINOR', rule_key: 'MISSING_TEST', file: 'commands/trekexecute.md', line: 1 + i,
title: `see ${cite} and ${cite}0`, detail: `${cite}2`, recommended_action: 'Add a test.',
}));
let reads = 0;
const real = fs.readFileSync;
mock.method(fs, 'readFileSync', function (p, ...rest) {
if (String(p).endsWith(`${'/'}commands/trekexecute.md`)) reads++;
return real.call(this, p, ...rest);
});
syncBuiltinESMExports();
let r;
try { r = validateFindings({ reviewer: 'r', findings }, { repoRoot: ROOT }); }
finally { mock.restoreAll(); syncBuiltinESMExports(); }
assert.equal(r.valid, true, JSON.stringify(r.errors));
assert.equal(reads, 1, `commands/trekexecute.md was read ${reads} times for one validation`);
});
test('review 630 r7 / d5ab7698: one isMain helper, and a CLI started through a symlinked path with a space does its work', () => {
const files = spawnSync('git', ['ls-files', 'lib', 'scripts', 'hooks'], { cwd: ROOT, encoding: 'utf8' }).stdout.split('\n').filter((p) => p.endsWith('.mjs'));
const local = files.filter((p) => p !== 'lib/util/is-main.mjs' &&
/function isMain\b|import\.meta\.url === `file:\/\/\$\{process\.argv\[1\]\}`/.test(read(p)));
assert.deepEqual(local, [], 'every CLI uses lib/util/is-main.mjs');
const base = mkdtempSync(join(tmpdir(), 'r630 r7 main '));
const link = join(base, 'voyage link');
try {
symlinkSync(ROOT, link);
// The first had the file:// guard (a silent exit 0 here), the second the realpath copy.
for (const cli of ['lib/validators/session-state-validator.mjs', 'lib/validators/plan-validator.mjs']) {
const r = spawnSync(process.execPath, [join(link, cli)], { encoding: 'utf8', input: '', timeout: 30_000 });
assert.equal(r.status, 2, `${cli} through "${link}": ${r.stdout}${r.stderr}`);
assert.match(r.stderr, /usage/i);
}
} finally { fs.rmSync(link, { force: true }); rmSync(base, { recursive: true, force: true }); }
});
test('review 630 r7 / 4f8a69e7: the guard writes its no-one-to-ask refusal once, on a direct condition', () => {
const src = read('hooks/scripts/pre-bash-executor.mjs');
assert.doesNotMatch(src, /const unattended = /);
assert.equal(src.split('permission mode where Claude Code asks nothing').length - 1, 1, 'one refusal text for both cases');
});
test('review 630 r7 / findings-schema.mjs:41: a triple backtick inside a JSON string does not end the block', async () => {
const { extractFindingsBlock } = await import('../../lib/review/findings-schema.mjs');
const { fillFindingIds } = await import('../../lib/review/fill-finding-ids.mjs');
const payload = { verdict: 'WARN', findings: [{ id: '@@ID_1@@', severity: 'MAJOR', rule_key: 'MISSING_TEST', file: 'a.mjs', line: 1,
title: 'x', detail: 'write the check as ```bash\nnpm test\n``` and run it', recommended_action: 'Add a test.' }] };
const text = `# Review\n\nProse with a \`\`\`json fence quoted in it.\n\n\`\`\`json\n${JSON.stringify(payload, null, 2)}\n\`\`\`\n`;
const block = extractFindingsBlock(text);
assert.match(block, /"recommended_action": "Add a test\."\s*\}\s*\]\s*\}\s*$/, `the block runs to its own closing fence, not to the backticks in a string: …${block.slice(-60)}`);
assert.deepEqual(JSON.parse(block), payload);
const r = fillFindingIds(text);
assert.equal(r.ok, true, JSON.stringify(r.errors));
assert.doesNotMatch(r.text, /@@ID_1@@/);
});
test('review 630 r7 / brief note 3: the prose says a git revision may start with @, as the screen allows', async () => {
const { allowedCommand } = await import('../../lib/verification/criteria-runner.mjs');
assert.equal(allowedCommand('git show @~1 --stat', { cwd: ROOT }).allowed, true);
assert.equal(allowedCommand('pytest @args.txt', { cwd: ROOT }).allowed, false, 'the control: a runner word with a leading @');
for (const rel of ['agents/brief-reviewer.md', 'commands/trekplan.md']) {
assert.match(read(rel), /`@~1`/, `${rel} names the git revision as the exception`);
}
});

29
tests/scripts/release-check.test.mjs
View file

@ -13,7 +13,7 @@
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, existsSync } from 'node:fs';
import { mkdtempSync, mkdirSync, readFileSync, writeFileSync, rmSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
@ -462,3 +462,30 @@ test('release-check r6 — a suite that never ends is stopped and read as not me
assert.equal(r.exit, 2);
} finally { fx.cleanup(); }
});
// ==== round 7 of the release review of 6.3.0 (2026-10-11), fixed in 6.3.1 ====
test('release-check r7 — a suite killed for too much output is not called a timeout → 2', () => {
const fx = fixture({ testScript: "node -e \"process.stdout.write('x'.repeat(200000))\"" });
try {
const r = mod.releaseCheck({ repo: fx.repo, review: fx.review, suiteMaxBuffer: 4096 });
const suite = r.checks.find((c) => c.id === 'suite');
assert.equal(suite.status, 'error', JSON.stringify(suite));
assert.doesNotMatch(suite.detail, /timeout/i, suite.detail);
assert.match(suite.detail, /output/i, suite.detail);
assert.equal(r.exit, 2);
} finally { fx.cleanup(); }
});
test('release-check r7 — a relative --review is read in --repo, like every other path → 0', () => {
const fx = fixture();
try {
mkdirSync(join(fx.repo, '.claude', 'projects', 'x'), { recursive: true });
writeFileSync(join(fx.repo, '.claude', 'projects', 'x', 'review.md'), readFileSync(fx.review, 'utf8'));
writeFileSync(join(fx.repo, '.git', 'info', 'exclude'), '.claude/\n');
// Run from the directory above the repo, as `--repo ../voyage` from a sibling would.
const r = spawnSync(process.execPath, [CHECK, '--repo', 'repo', '--review', '.claude/projects/x/review.md'], { cwd: fx.base, encoding: 'utf8' });
assert.doesNotMatch(r.stdout, /does not exist/, r.stdout);
assert.equal(r.status, 0, r.stdout + r.stderr);
} finally { fx.cleanup(); }
});