voyage/lib/review/engine-findings.mjs
Kjell Tore Guttormsen 29b4ac91b3
fix: round 7 of the release review of 6.3.0, for 6.3.1
Round 7 (516dffa..45aa9e1, WARN: 12 MAJOR, 0 BLOCKER) and the fence defect
found while writing its review.md. Each fix began with a test that failed on
0f400d3 (v6.3.0); suite 2208 (2206/0/2), 14 tests more.

- Tests-only (operator 2026-10-11): the plan's Files: alone decides, and the
  text says so (red-proof, trekexecute Sub-step B, step-implementer,
  HANDOVER-CONTRACTS). red-proof no longer reads the step's diff, so a change
  already in the tree when the step began no longer counts against it, and a
  plan placed outside .claude/ reads the same. The round-6 test that had the
  diff overrule the plan is rewritten to the new rule.
- The Files: reader keeps a plain file beside a backticked path
  (", Dockerfile", "the CLI entry src/cli.mjs", "and Makefile"). Measured on
  the 520 Files: fields in 182 plans: no field reads differently.
- Red proof reports a command its timeout killed as killed after the timeout,
  not "could not start"; the release check tells a timeout from output past
  its buffer and from another signal. One reading of a stop
  (criteria-runner stoppedBy) for red proof, the criteria runner and the
  release check.
- A trailing `# comment` is dropped in a backticked check too, and in a
  command passed as written, so the words screened are the words run.
- The release check reads a relative --review in --repo.
- The 15-minute timeout is defined once (criteria-runner.mjs).
- The findings check reads each cited file once per validation.
- One isMain helper (lib/util/is-main.mjs) for every CLI: the 31 copies of
  the realpath guard and the 20 modules still on the file:// guard, which
  exited 0 having done nothing under a symlink or a path with a space.
- The Bash guard writes its no-one-to-ask refusal once, on a direct condition.
- The findings check ends a ```json block only at a fence on its own line, so
  three backticks inside a JSON string no longer end it (also in 6.2.0).
- Brief-conformance notes: a test for test-lock green on exit 127 (it
  passed at once; it fails when 127 is read as could-not-run); the prose now
  names the git revision `@~1` as the one leading-@ exception.

Not changed: the example plan's steps 1-2 keep their tests-only warning
(operator decision 2026-10-09; the r5 test records it). Brief note 4 (a
Regression: whose runner is not installed goes to the fix rounds) is a
routing question for the operator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 09:11:57 +02:00

176 lines
8.8 KiB
JavaScript

// lib/review/engine-findings.mjs
// Claude Code's review as the engine (ordre 5, P11 — wrap, not a new engine).
//
// /trekreview Phase 4.7 starts Claude Code's own `/code-review` and
// `/security-review` through the Skill tool (never with `--fix`, never anything
// that edits code). Their findings come back as text: per finding a file
// location, a one-sentence summary, a failure scenario and an optional category
// slug (code.claude.com/docs/en/tools-reference.md, `ReportFindings`). The
// orchestrator transcribes them into a small JSON list; THIS module turns that
// list into a reviewer payload the coordinator already understands, so the
// engine's findings meet Voyage's brief control in the same dedup + verdict.
//
// Input (one engine per call):
// { "engine": "code-review" | "security-review",
// "findings": [ { "file": "...", "line": 12, "summary": "...",
// "failure_scenario": "...", "category": "correctness",
// "fix": "<optional: the fix the engine itself proposed>",
// "rule_key": "<optional narrower catalogue key>" } ] }
//
// Output: { reviewer, findings: [...catalogue-conformant...], out_of_scope: [...] }
// - rule_key: the given one when it is in the catalogue AND at least as
// severe as the engine's own key, else CODE_REVIEW_DEFECT (MAJOR) /
// SECURITY_REVIEW_VULNERABILITY (BLOCKER). A "narrower" key can never
// lower an engine finding's tier. One exception, decided by the engine's
// own category and not by the transcriber: a /code-review finding in an
// IMPROVEMENT_CATEGORIES category is CODE_REVIEW_IMPROVEMENT (MINOR), since
// /code-review reports no severity and a simplification proposal is not a
// MAJOR defect (voyage-lekerepo trial 2, 2026-10-08).
// - recommended_action: the engine's own `fix` when the transcription
// carries one; otherwise a test-first step built from the failure
// scenario. Never the scenario dressed as a remedy (trial 2: /trekplan
// would have taken failure scenarios as step goals).
// - File paths are made repo-relative (an absolute path under `repoRoot`
// loses that prefix) before they are compared with the scope.
// - A finding on a file outside the review scope is NOT dropped silently: it
// lands in `out_of_scope`, which the coordinator lists under Declined to
// Judge (`engine:out-of-scope`). It forbids ALLOW unless it is a MINOR or
// SUGGESTION from /code-review (coordinator-contract.mjs runContract);
// `/security-review` diffs against origin's default branch, which can be
// wider than the review's SHA range.
// - An empty scope list is an error, never "everything is out of scope".
//
// CLI: node lib/review/engine-findings.mjs [--scope <file-list.txt>] [--repo-root <dir>] <engine-findings.json>
// exit 0 = payload on stdout; 1 = input invalid (errors on stderr); 2 = could not run.
import { readFileSync, existsSync } from 'node:fs';
import { computeFindingId } from '../parsers/finding-id.mjs';
import { RULE_KEYS, getRule, SEVERITY_VALUES } from './rule-catalogue.mjs';
import { validateFindings } from './findings-schema.mjs';
import { isMain } from '../util/is-main.mjs';
export const ENGINES = Object.freeze({
'code-review': Object.freeze({
reviewer: 'claude-code-review',
rule_key: 'CODE_REVIEW_DEFECT',
brief_ref: 'NFR — code correctness (Claude Code /code-review)',
}),
'security-review': Object.freeze({
reviewer: 'claude-security-review',
rule_key: 'SECURITY_REVIEW_VULNERABILITY',
brief_ref: 'NFR — security (Claude Code /security-review)',
}),
});
function normPath(p, repoRoot) {
let out = String(p ?? '').trim().replace(/\\/g, '/');
if (repoRoot) {
const root = String(repoRoot).replace(/\\/g, '/').replace(/\/+$/, '') + '/';
if (out.startsWith(root)) out = out.slice(root.length);
}
return out.replace(/^(\.\/)+/, '');
}
function parseLine(v) {
if (Number.isInteger(v) && v >= 0) return v;
if (typeof v === 'string' && /^\d+$/.test(v.trim())) return Number(v.trim());
return 0;
}
const tier = (key) => SEVERITY_VALUES.indexOf(getRule(key).severity);
/** /code-review categories that name an improvement, not a defect. */
export const IMPROVEMENT_CATEGORIES = Object.freeze(new Set(['simplification', 'efficiency', 'reuse', 'style', 'readability']));
/**
* Convert one engine's transcribed findings into a coordinator payload.
* @param {{ engine: string, findings: object[] }} input
* @param {{ scopeFiles?: string[], repoRoot?: string }} [opts]
* @returns {{ ok: boolean, errors: string[], payload?: { reviewer: string, findings: object[] }, out_of_scope?: object[] }}
*/
export function toReviewerPayload(input, opts = {}) {
const errors = [];
const spec = ENGINES[input?.engine];
if (!spec) return { ok: false, errors: [`engine must be one of ${Object.keys(ENGINES).join('|')}, got ${JSON.stringify(input?.engine)}`] };
if (!Array.isArray(input.findings)) return { ok: false, errors: ['findings must be an array'] };
if (opts.scopeFiles && opts.scopeFiles.length === 0) {
return { ok: false, errors: ['scope list is empty — the review scope is unknown, so no engine finding can be placed'] };
}
const scope = opts.scopeFiles ? new Set(opts.scopeFiles.map((f) => normPath(f, opts.repoRoot))) : null;
const findings = [];
const out_of_scope = [];
input.findings.forEach((f, i) => {
const file = normPath(f?.file, opts.repoRoot);
const summary = String(f?.summary ?? '').trim();
if (!file) errors.push(`findings[${i}].file is empty`);
if (!summary) errors.push(`findings[${i}].summary is empty`);
if (!file || !summary) return;
const line = parseLine(f.line);
const narrower = typeof f.rule_key === 'string' && RULE_KEYS.has(f.rule_key)
&& tier(f.rule_key) <= tier(spec.rule_key);
const improvement = input.engine === 'code-review'
&& IMPROVEMENT_CATEGORIES.has(String(f.category ?? '').trim().toLowerCase());
const engineKey = improvement ? 'CODE_REVIEW_IMPROVEMENT' : spec.rule_key;
const rule_key = narrower ? f.rule_key : engineKey;
const scenario = String(f.failure_scenario ?? '').trim();
const fix = String(f.fix ?? '').trim();
const loc = line > 0 ? `${file}:${line}` : file;
const out = {
id: computeFindingId(file, line, rule_key),
severity: getRule(rule_key).severity,
rule_key,
file,
line,
brief_ref: spec.brief_ref,
title: summary,
detail: [String(f.failure_scenario ?? '').trim(), f.category ? `(engine category: ${f.category})` : '']
.filter(Boolean).join(' ') || summary,
// An action, not the title again: `Resolve: <title>` read as "restating
// the title" to the coordinator's Actionability filter, and 8 of 8
// engine findings were set aside (voyage-lekerepo trial, 2026-10-06).
recommended_action: fix
|| (scenario
? `Write a test that reproduces this at ${loc} (${scenario}), then change ${loc} until it passes`
: `Write a test that shows the defect at ${loc} that ${input.engine === 'security-review' ? '/security-review' : '/code-review'} reported, then change ${loc} until it passes`),
engine: input.engine,
};
if (scope && !scope.has(file)) out_of_scope.push(out);
else findings.push(out);
});
if (errors.length > 0) return { ok: false, errors };
const payload = { reviewer: spec.reviewer, findings };
const v = validateFindings(payload);
if (!v.valid) return { ok: false, errors: v.errors.map((e) => `${e.code}: ${e.message}`) };
return { ok: true, errors: [], payload, out_of_scope };
}
if (isMain(import.meta.url)) {
const args = process.argv.slice(2);
const valueOf = (name) => { const i = args.indexOf(name); return i === -1 ? null : args[i + 1]; };
const scopePath = valueOf('--scope');
const repoRoot = valueOf('--repo-root');
const flagValues = new Set([scopePath, repoRoot].filter(Boolean));
const filePath = args.find((a) => !a.startsWith('--') && !flagValues.has(a));
if (!filePath || !existsSync(filePath) || (scopePath && !existsSync(scopePath))) {
process.stderr.write('Usage: engine-findings.mjs [--scope <file-list.txt>] [--repo-root <dir>] <engine-findings.json>\n');
process.exit(2);
}
let input;
let scopeFiles;
try {
input = JSON.parse(readFileSync(filePath, 'utf-8'));
if (scopePath) scopeFiles = readFileSync(scopePath, 'utf-8').split('\n').map((l) => l.trim()).filter(Boolean);
} catch (e) {
process.stderr.write(`engine-findings: could not read input: ${e.message}\n`);
process.exit(2);
}
const r = toReviewerPayload(input, { scopeFiles, repoRoot: repoRoot ?? undefined });
if (!r.ok) {
for (const e of r.errors) process.stderr.write(` ERROR ${e}\n`);
process.exit(1);
}
process.stdout.write(JSON.stringify({ ...r.payload, out_of_scope: r.out_of_scope }, null, 2) + '\n');
process.exit(0);
}