open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

fix(review): close the gate's fail-open paths an independent review found

Browse source 
An adversarial read of d88245c reproduced 15 problems; all fixed here, each
with a test that fails on the old code.

- verdict-gate: a greener verdict must be backed by the LAST fence being a
  `json` block of the right shape, with no BLOCKER in the JSON (any case),
  in `## Findings (BLOCKER)`, or a differing finding-id list; two verdict
  lines and more than one file argument are exit 2. ({}, null, lowercase
  "blocker", a trailing ```jsonc and a later json fence all gave exit 0.)
- engine-findings: a transcribed "narrower" key can no longer lower an
  engine tier; absolute paths under --repo-root match the scope; "12" lines
  parse; an empty scope is an error; out-of-scope engine findings are
  unverified (`engine:out-of-scope`) and forbid ALLOW.
- recurrence: the catch-all engine keys and keys normalised from unknown
  ones are never proposed; the same unfixed finding in two iterations is not
  a recurrence; proposals are validated before they are offered; \r/U+2028
  and damaged markers are refused; writes go through a symlinked CLAUDE.md.
  Prose points --repo at the session directory, not the git toplevel.
- Workflow substrate: engines are counted from what arrived, and the
  verdict's floor is recomputed from the coordinator's findings; now tested
  by running the script with stubbed Workflow globals. Two literal NUL bytes
  written as \u0000 (git saw the file as binary).
- Trimming keeps Pass 1's "Also flagged by" provenance; `## Declined to
  Judge` and remediation steps are required from review_version 1.1 only
  (N-1); the zero-diff review now carries the JSON block the gate needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-06 20:03:16 +02:00
commit 40f6ae1177
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
17 changed files with 581 additions and 78 deletions
Download patch file Download diff file Expand all files Collapse all files

4
agents/review-coordinator.md
View file

@ -54,6 +54,10 @@ You will receive a prompt containing:
- **Expected reviewers** — the names that were due, engines included, and
each engine recorded as not run with its reason. A name that did not report
forbids `ALLOW` (Pass 4).
- **Engine findings outside the review scope** — the `out_of_scope` list from
`lib/review/engine-findings.mjs`. Each one is `unverified`
(`engine:out-of-scope`): a real claim on a file this review did not cover.
List it under `## Declined to Judge`; it forbids `ALLOW`.
- **Triage map** — `{file → deep-review|summary-only|skip, reason}` from
the /trekreview triage gate.
- **Brief metadata** — `task`, `slug`, `project_dir`, `brief_path` from

62
commands/trekreview.md
View file

@ -36,7 +36,7 @@ part that stops (Phase 5–6); the verdict is written in `review.md` and one
command decides on it (`lib/review/verdict-gate.mjs`, Phase 8: 0 = ALLOW/WARN,
1 = BLOCK, 2 = could not decide, never green). No finding is struck for
length. A finding class seen in two reviews becomes one proposal for the
project's memory, approved line by line (Phase 9.5). Order credited to Boris
project's memory, approved line by line (Phase 8.7). Order credited to Boris
Cherny's public workflow (engine → brief control); the gate to Kiro and Spec
Kit — see `docs/HANDOVER-CONTRACTS.md` § Handover 6.
@ -163,10 +163,15 @@ explicitly in the Coverage table.
If the file count is `0`, write a minimal review.md per the template:
`verdict: ALLOW`, `findings: []` (the empty list is the one inline form the
parser reads — a bare `findings:` is null and fails validation), and the
three required sections, each saying "No diff between {before_sha} and
{after_sha}; nothing to review." Skip Phases 4–7. Continue to Phase 8
(validate + stats).
parser reads — a bare `findings:` is null and fails validation), the
required sections (`## Executive Summary`, `## Coverage`, `## Declined to
Judge` with `None.` and `Trimmed: 0`, `## Remediation Summary`), each saying
"No diff between {before_sha} and {after_sha}; nothing to review.", and the
trailing block `{"verdict": "ALLOW", "counts": {"BLOCKER": 0, "MAJOR": 0,
"MINOR": 0, "SUGGESTION": 0}, "findings": []}` — without it the verdict gate
cannot back the ALLOW and exits 2. There is nothing for the Phase 4.7 engine
to review, so its absence does not withhold ALLOW here. Skip Phases 4–7.
Continue to Phase 8 (validate + stats).
## Phase 4 — Triage gate (deterministic path-pattern classifier)
@ -311,16 +316,26 @@ drop an engine finding on the way. Then convert it:
git diff --name-only "$before_sha".."$after_sha" > "{project_dir}/.review-scope.txt"
git diff --name-only HEAD >> "{project_dir}/.review-scope.txt"
node ${CLAUDE_PLUGIN_ROOT}/lib/review/engine-findings.mjs \
--scope "{project_dir}/.review-scope.txt" "{project_dir}/.review-engine-code-review.json"
--scope "{project_dir}/.review-scope.txt" --repo-root "$(git rev-parse --show-toplevel)" \
"{project_dir}/.review-engine-code-review.json"
```
Exit 0 prints a reviewer payload (`claude-code-review` /
`claude-security-review`) for Phase 6 plus `out_of_scope`: engine findings on
files outside the review's SHA range (`/security-review` diffs against
origin's default branch, which can be wider). Those go in the Coverage
section, one line each — never silently dropped. Exit 1 = the transcription is
malformed: fix it from the engine's own text and re-run. Exit 2 = could not
run.
origin's default branch, which can be wider). Pass `out_of_scope` to the
coordinator as well: each one is `unverified` (`engine:out-of-scope`), is
listed under `## Declined to Judge` and in Coverage, and forbids `ALLOW` —
never silently dropped. Absolute paths under the repo root are made relative
before the scope match (`--repo-root`). Exit 1 = the transcription is
malformed, or the scope list is empty: fix it from the engine's own text (or
the scope from Phase 3) and re-run; never feed a payload that did not pass.
Exit 2 = could not run — record that engine as not run.
A "narrower" `rule_key` you transcribe is used only when its catalogue tier is
at least the engine's own (MAJOR for `/code-review`, BLOCKER for
`/security-review`); the adapter refuses to let a transcription lower an
engine finding.
**An engine that did not run** (Claude Code older than v2.1.246, the Skill
call refused, no `origin` remote, an `ambiguous argument` error, or no findings
@ -576,8 +591,12 @@ node ${CLAUDE_PLUGIN_ROOT}/lib/review/verdict-gate.mjs "{review_path}"
```
Exit 0 = ALLOW or WARN; exit 1 = BLOCK; exit 2 = the gate could not decide
(missing verdict, a verdict the file contradicts, unreadable file) — **never
green**. Capture the exit code as `gate_exit` and print it in Phase 9 exactly
(missing verdict, a verdict the file contradicts — in its JSON block, its
`## Findings (BLOCKER)` section or its finding-id list — a JSON block that is
not the last fence or has the wrong shape, two verdict lines, unreadable
file) — **never green**. It takes exactly one file. On exit 0 with WARN,
`--json` also returns `warn_reasons` (the JSON block's `allow_blocked_by`);
print them. Capture the exit code as `gate_exit` and print it in Phase 9 exactly
as returned. This is the command that later stops a merge or a release on
BLOCK; `/trekreview` reports it and never overrides it.
@ -628,9 +647,14 @@ repo already has (every `.claude/projects/*/review.md`, plus earlier committed
versions of them) — no new store.
```bash
node ${CLAUDE_PLUGIN_ROOT}/lib/review/recurrence.mjs propose --repo "$(git rev-parse --show-toplevel)"
node ${CLAUDE_PLUGIN_ROOT}/lib/review/recurrence.mjs propose --repo "$PWD"
```
`--repo` is the session's working directory — the directory `project_dir`
paths are relative to, and the project Claude Code loads `.claude/rules/`
from. Not the git toplevel: for a project in a subdirectory that would read
no reviews and write the lessons in the wrong place.
It prints `{"proposals": [...]}` (exit 0) or exits 2 (could not run — say so
in the summary; never read 2 as "nothing recurred"). Each proposal names its
`rule_key`, its route and its evidence (the finding IDs from two reviews), and
@ -652,7 +676,7 @@ to a line it was not shown.
On yes:
```bash
node ${CLAUDE_PLUGIN_ROOT}/lib/review/recurrence.mjs apply --repo "$(git rev-parse --show-toplevel)" \
node ${CLAUDE_PLUGIN_ROOT}/lib/review/recurrence.mjs apply --repo "$PWD" \
--rule-key <RULE_KEY> [--line "<the operator's edited line>"]
```
@ -661,8 +685,14 @@ It writes ONLY between Voyage's own markers in
gives it a one-line pointer once (never the lesson itself). Exit 1 with
`over-budget` means the managed block is full: tell the operator the lessons
belong in a skill now — that is the operator's decision, not this command's.
Exit 1 with `invalid-line` means an edited line lacks the three parts: show
why and ask again. Report every outcome in the summary; when no class
Exit 1 with `invalid-line` means the line was refused by the schema check
(missing one of the three parts, a line break or control character, marker
text, or over 700 characters) — show the `detail` and ask again. Exit 1 with
`markers-damaged` means Voyage's markers in the lessons file are not one
start before one end: say so; the operator fixes the file by hand. A
finding class only counts when two reviews hold two DIFFERENT findings of it
— one unfixed finding seen again is a persisting defect, not a lesson — and
the two catch-all engine keys are never proposed. Report every outcome in the summary; when no class
recurred, say "no recurring finding class" — do not pad.
Borrowed: routing (Pocock `retro`), the three-part line (ai-native-sdlc),

6
docs/HANDOVER-CONTRACTS.md
View file

@ -336,7 +336,7 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
**Consumer:** `/trekplan` Phase 1 when `--brief review.md` is supplied and the consumer detects `type: trekreview` in frontmatter. The plan command branches into a remediation-plan path: BLOCKER + MAJOR findings become plan goals, the produced `plan.md` carries a `source_findings: [<id>, ...]` frontmatter list as the audit trail back to the consumed findings. MINOR + SUGGESTION are skipped for plan-input. Since 1.1 every BLOCKER is also written as a numbered `### Step R<n>` under `## Remediation Steps` (Files · Source finding · Test first · Verify · On failure), so a consumer can append it as a plan step without rewording it.
**Second consumer — the verdict gate (K7, 1.1):** `lib/review/verdict-gate.mjs <review.md>` reads the frontmatter `verdict` and exits 0 = ALLOW/WARN, 1 = BLOCK, 2 = could not decide. Exit 2 is never green: a missing file, a missing or unknown verdict, and a file whose trailing JSON block or BLOCKER findings contradict a greener verdict all land there. Whatever stops a merge or a release on a review reads this exit code, never a summary. Borrowed: the verdict goes to a file and a command gates on it (Kiro `feature-pipeline`); the executor reads the verdict but never writes it (Spec Kit).
**Second consumer — the verdict gate (K7, 1.1):** `lib/review/verdict-gate.mjs <review.md>` reads the frontmatter `verdict` and exits 0 = ALLOW/WARN, 1 = BLOCK, 2 = could not decide. Exit 2 is never green: a missing file, a missing or unknown verdict, two verdict lines, a trailing JSON block that is not the last fence or lacks `verdict`/`counts`/`findings`, and a file whose JSON block, `## Findings (BLOCKER)` section or finding-id list contradicts a greener verdict all land there. Exactly one file per call. Whatever stops a merge or a release on a review reads this exit code, never a summary. Borrowed: the verdict goes to a file and a command gates on it (Kiro `feature-pipeline`); the executor reads the verdict but never writes it (Spec Kit).
**Path conventions:**
- Project-dir mode (recommended): `{project_dir}/review.md` (one per review iteration; subsequent runs overwrite atomically).
@ -369,8 +369,8 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
Optional but standard sections: `## Findings (BLOCKER)`, `## Findings (MAJOR)`, `## Findings (MINOR)`, `## Findings (SUGGESTION)`. The `## Coverage` section enumerates which files were deep-reviewed, summary-only, or skipped (with reason) — this is how the triage gate stays honest and avoids Copilot-style silent skips; engine findings on files outside the review's SHA range are listed there too.
Since 1.1 (ordre 5):
- `## Remediation Steps` — required when there is a BLOCKER: one `### Step R<n>` per BLOCKER carrying `Source finding: <id>` (strict error `REVIEW_MISSING_REMEDIATION_STEP`, soft warning). Steps are appended and numbered, never reworded across iterations (Spec Kit `/converge`).
- `## Declined to Judge` — written by the coordinator: every finding set aside without a ruling, with its reason, plus `Trimmed: N`. **No finding is struck for length**: only MINOR/SUGGESTION text is trimmed, and counted (`trimmed` in the JSON block); a long BLOCKER/MAJOR keeps every character.
- `## Remediation Steps` — required at `review_version` ≥ 1.1 when there is a BLOCKER: one `### Step R<n>` per BLOCKER carrying `Source finding: <id>` (strict error `REVIEW_MISSING_REMEDIATION_STEP`, soft warning). Steps are appended and numbered, never reworded across iterations (Spec Kit `/converge`).
- `## Declined to Judge` — required at `review_version` ≥ 1.1 (strict error, soft warning; 1.0 reviews exempt): every finding set aside without a ruling, with its reason (including engine findings outside the review scope, `engine:out-of-scope`), plus `Trimmed: N`. **No finding is struck for length**: only MINOR/SUGGESTION text is trimmed, and counted (`trimmed` in the JSON block); a long BLOCKER/MAJOR keeps every character.
- Findings from Claude Code's own `/code-review` and `/security-review` enter under the catalogue keys `CODE_REVIEW_DEFECT` (MAJOR) and `SECURITY_REVIEW_VULNERABILITY` (BLOCKER), or a narrower key, with `reviewer` `claude-code-review` / `claude-security-review`.
**Validation strategy:**

26
lib/review/coordinator-contract.mjs
View file

@ -77,6 +77,7 @@ export const REFUTING_REASONS = Object.freeze(new Set([
*/
export const UNVERIFIED_REASONS = Object.freeze([
'actionability:empty',
'engine:out-of-scope',
'unknown-rule_key',
'file-existence:indeterminate',
]);
@ -214,6 +215,21 @@ function trimText(text, max) {
return text.slice(0, max - TRIM_MARKER.length) + TRIM_MARKER;
}
/**
* Trim a detail but keep Pass 1's "Also flagged by …" provenance lines: the
* reviewer's own text is what gets cut. When the provenance alone is too long
* it is cut too — `raised_by` still names every reviewer.
*/
function trimDetail(detail, max) {
const at = detail.indexOf('\nAlso flagged by ');
if (at === -1) return trimText(detail, max);
const head = detail.slice(0, at);
const tail = detail.slice(at);
const room = max - tail.length - TRIM_MARKER.length;
if (room < 40) return trimText(detail, max);
return head.slice(0, room) + TRIM_MARKER + tail;
}
/**
* Pass 2 — HubSpot Judge (deterministic subset): succinctness and
* actionability (recommended_action, when present, must be a non-empty
@ -250,7 +266,7 @@ export function judgeFilter(findings) {
const out = { ...f };
const fields = [];
if (longTitle) { out.title = trimText(title, JUDGE_TITLE_MAX); fields.push('title'); }
if (longDetail) { out.detail = trimText(detail, JUDGE_DETAIL_MAX); fields.push('detail'); }
if (longDetail) { out.detail = trimDetail(detail, JUDGE_DETAIL_MAX); fields.push('detail'); }
out.trimmed_fields = fields;
kept.push(out);
trimmed.push(out);
@ -357,7 +373,7 @@ export function computeVerdict(findings, options = {}) {
* forbids ALLOW. Do not iterate both and count twice.
*
* @param {Array<{reviewer?: string, findings: object[]}>} reviewerPayloads
* @param {{ expectedReviewers?: string[] }} [options]
* @param {{ expectedReviewers?: string[], engineOutOfScope?: object[] }} [options]
* `declined_to_judge` lists every unverified finding with its reason, and
* `trimmed_count` counts the MINOR/SUGGESTION findings whose text was trimmed
* (kept, judged, and counted — never removed for length).
@ -369,7 +385,11 @@ export function runContract(reviewerPayloads, options = {}) {
const deduped = dedupByTriplet(ingested);
const judged = judgeFilter(deduped);
const reasoned = reasonablenessFilter(judged.kept);
const unverified = [...judged.unverified, ...reasoned.unverified];
// Engine findings on files outside the review's SHA range (engine-findings.mjs
// `out_of_scope`): real claims nobody here judged, so they forbid ALLOW.
const outOfScope = (options.engineOutOfScope ?? [])
.map((f) => ({ ...f, suppressed_reason: 'engine:out-of-scope' }));
const unverified = [...judged.unverified, ...reasoned.unverified, ...outOfScope];
// What the coordinator set aside without ruling on it, one line each, so a
// reader sees every finding that did not reach a verdict tier. Borrowed:
// "Declined to judge … nothing you set aside is dropped silently"

64
lib/review/engine-findings.mjs
View file

@ -17,20 +17,26 @@
// "rule_key": "<optional narrower catalogue key>" } ] }
//
// Output: { reviewer, findings: [...catalogue-conformant...], out_of_scope: [...] }
// - rule_key: the given one when it is in the catalogue, else
// CODE_REVIEW_DEFECT (MAJOR) / SECURITY_REVIEW_VULNERABILITY (BLOCKER).
// - rule_key: the given one when it is in the catalogue AND at least as
// severe as the engine's own key, else CODE_REVIEW_DEFECT (MAJOR) /
// SECURITY_REVIEW_VULNERABILITY (BLOCKER). A "narrower" key can never
// lower an engine finding's tier.
// - File paths are made repo-relative (an absolute path under `repoRoot`
// loses that prefix) before they are compared with the scope.
// - A finding on a file outside the review scope is NOT dropped silently: it
// lands in `out_of_scope` for the Coverage section (`/security-review`
// diffs against origin's default branch, which can be wider than the
// review's SHA range).
// lands in `out_of_scope`, which the coordinator counts as unverified
// (`engine:out-of-scope`) — it forbids ALLOW and is listed under
// Declined to Judge (`/security-review` diffs against origin's default
// branch, which can be wider than the review's SHA range).
// - An empty scope list is an error, never "everything is out of scope".
//
// CLI: node lib/review/engine-findings.mjs [--scope <file-list.txt>] <engine-findings.json>
// CLI: node lib/review/engine-findings.mjs [--scope <file-list.txt>] [--repo-root <dir>] <engine-findings.json>
// exit 0 = payload on stdout; 1 = input invalid (errors on stderr); 2 = could not run.
import { readFileSync, existsSync, realpathSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import { computeFindingId } from '../parsers/finding-id.mjs';
import { RULE_KEYS, getRule } from './rule-catalogue.mjs';
import { RULE_KEYS, getRule, SEVERITY_VALUES } from './rule-catalogue.mjs';
import { validateFindings } from './findings-schema.mjs';
export const ENGINES = Object.freeze({
@ -46,14 +52,27 @@ export const ENGINES = Object.freeze({
}),
});
function normPath(p) {
return String(p ?? '').replace(/^\.\//, '').trim();
function normPath(p, repoRoot) {
let out = String(p ?? '').trim().replace(/\\/g, '/');
if (repoRoot) {
const root = String(repoRoot).replace(/\\/g, '/').replace(/\/+$/, '') + '/';
if (out.startsWith(root)) out = out.slice(root.length);
}
return out.replace(/^(\.\/)+/, '');
}
function parseLine(v) {
if (Number.isInteger(v) && v >= 0) return v;
if (typeof v === 'string' && /^\d+$/.test(v.trim())) return Number(v.trim());
return 0;
}
const tier = (key) => SEVERITY_VALUES.indexOf(getRule(key).severity);
/**
* Convert one engine's transcribed findings into a coordinator payload.
* @param {{ engine: string, findings: object[] }} input
* @param {{ scopeFiles?: string[] }} [opts]
* @param {{ scopeFiles?: string[], repoRoot?: string }} [opts]
* @returns {{ ok: boolean, errors: string[], payload?: { reviewer: string, findings: object[] }, out_of_scope?: object[] }}
*/
export function toReviewerPayload(input, opts = {}) {
@ -61,18 +80,23 @@ export function toReviewerPayload(input, opts = {}) {
const spec = ENGINES[input?.engine];
if (!spec) return { ok: false, errors: [`engine must be one of ${Object.keys(ENGINES).join('|')}, got ${JSON.stringify(input?.engine)}`] };
if (!Array.isArray(input.findings)) return { ok: false, errors: ['findings must be an array'] };
const scope = opts.scopeFiles ? new Set(opts.scopeFiles.map(normPath)) : null;
if (opts.scopeFiles && opts.scopeFiles.length === 0) {
return { ok: false, errors: ['scope list is empty — the review scope is unknown, so no engine finding can be placed'] };
}
const scope = opts.scopeFiles ? new Set(opts.scopeFiles.map((f) => normPath(f, opts.repoRoot))) : null;
const findings = [];
const out_of_scope = [];
input.findings.forEach((f, i) => {
const file = normPath(f?.file);
const file = normPath(f?.file, opts.repoRoot);
const summary = String(f?.summary ?? '').trim();
if (!file) errors.push(`findings[${i}].file is empty`);
if (!summary) errors.push(`findings[${i}].summary is empty`);
if (!file || !summary) return;
const line = Number.isInteger(f.line) && f.line >= 0 ? f.line : 0;
const rule_key = typeof f.rule_key === 'string' && RULE_KEYS.has(f.rule_key) ? f.rule_key : spec.rule_key;
const line = parseLine(f.line);
const narrower = typeof f.rule_key === 'string' && RULE_KEYS.has(f.rule_key)
&& tier(f.rule_key) <= tier(spec.rule_key);
const rule_key = narrower ? f.rule_key : spec.rule_key;
const out = {
id: computeFindingId(file, line, rule_key),
severity: getRule(rule_key).severity,
@ -107,11 +131,13 @@ function isMain() {
if (isMain()) {
const args = process.argv.slice(2);
const scopeIdx = args.indexOf('--scope');
const scopePath = scopeIdx === -1 ? null : args[scopeIdx + 1];
const filePath = args.find((a, i) => !a.startsWith('--') && (scopeIdx === -1 || i !== scopeIdx + 1));
const valueOf = (name) => { const i = args.indexOf(name); return i === -1 ? null : args[i + 1]; };
const scopePath = valueOf('--scope');
const repoRoot = valueOf('--repo-root');
const flagValues = new Set([scopePath, repoRoot].filter(Boolean));
const filePath = args.find((a) => !a.startsWith('--') && !flagValues.has(a));
if (!filePath || !existsSync(filePath) || (scopePath && !existsSync(scopePath))) {
process.stderr.write('Usage: engine-findings.mjs [--scope <file-list.txt>] <engine-findings.json>\n');
process.stderr.write('Usage: engine-findings.mjs [--scope <file-list.txt>] [--repo-root <dir>] <engine-findings.json>\n');
process.exit(2);
}
let input;
@ -123,7 +149,7 @@ if (isMain()) {
process.stderr.write(`engine-findings: could not read input: ${e.message}\n`);
process.exit(2);
}
const r = toReviewerPayload(input, { scopeFiles });
const r = toReviewerPayload(input, { scopeFiles, repoRoot: repoRoot ?? undefined });
if (!r.ok) {
for (const e of r.errors) process.stderr.write(` ERROR ${e}\n`);
process.exit(1);

68
lib/review/recurrence.mjs
View file

@ -2,7 +2,11 @@
// P5 — the project's memory, fed by review (ordre 5).
//
// A finding class (the same catalogue `rule_key`) that shows up in TWO review.md
// files in the target repo becomes ONE proposal; one sighting is no proposal.
// files in the target repo, as two DIFFERENT findings (distinct finding IDs),
// becomes ONE proposal; one sighting is no proposal, and one unfixed finding
// seen again in the next iteration is a persisting defect, not a recurrence.
// Catch-all keys (route null) and findings whose key was normalised from an
// unknown one (`original_rule_key`) are never counted.
// The history is the review.md files the repo already has — the current
// `.claude/projects/*/review.md` plus, where git tracks them, their earlier
// committed versions. No new store is created.
@ -14,7 +18,7 @@
//
// Writes: only between Voyage's own markers, under a size budget; past the
// budget the write is refused and the lesson belongs in a skill (operator's
// call). Every line is approved by the operator first (/trekreview Phase 9.5,
// call). Every line is approved by the operator first (/trekreview Phase 8.7,
// AskUserQuestion) — this module never decides that on its own.
//
// Borrowed (credited, MIT where it applies):
@ -33,7 +37,7 @@
// → {"written":true|false, "reason":…} on stdout; exit 0 written, 1 refused, 2 could not run
import {
readFileSync, writeFileSync, renameSync, existsSync, mkdirSync, readdirSync, statSync, realpathSync,
readFileSync, writeFileSync, renameSync, existsSync, mkdirSync, readdirSync, statSync, realpathSync, lstatSync,
} from 'node:fs';
import { join, relative, dirname } from 'node:path';
import { execFileSync } from 'node:child_process';
@ -142,10 +146,18 @@ function lessonsPath(repo) {
return join(repo, LESSONS_DIR, LESSONS_FILE_NAME);
}
function count(text, needle) {
return text.split(needle).length - 1;
}
/** The managed block, null when absent, or { damaged: true } when the markers are not exactly one START before one END. */
function readBlock(text) {
const starts = count(text, MARK_START);
const ends = count(text, MARK_END);
if (starts === 0 && ends === 0) return null;
const start = text.indexOf(MARK_START);
const end = text.indexOf(MARK_END);
if (start === -1 || end === -1 || end < start) return null;
if (starts !== 1 || ends !== 1 || end < start) return { damaged: true };
return { start, end, inner: text.slice(start + MARK_START.length, end) };
}
@ -158,7 +170,7 @@ export function writtenClasses(repo) {
const p = lessonsPath(repo);
if (!existsSync(p)) return new Set();
const block = readBlock(readFileSync(p, 'utf-8'));
if (!block) return new Set();
if (!block || block.damaged) return new Set();
const keys = new Set();
for (const line of blockEntries(block.inner)) {
const m = line.match(/^- \[([A-Z_]+)\]/);
@ -170,7 +182,7 @@ export function writtenClasses(repo) {
// ---- proposals --------------------------------------------------------------
function clip(s, n) {
const t = String(s ?? '').replace(/\s+/g, ' ').trim();
const t = String(s ?? '').replace(/<!--|-->/g, '').replace(/[\u0000-\u001f\u007f\u2028\u2029\s]+/g, ' ').trim();
return t.length > n ? `${t.slice(0, n - 1)}…` : t;
}
@ -182,9 +194,13 @@ function composeLine(rule, evidence) {
const how = rule.memory_route === 'check'
? CHECK_HINTS[rule.rule_key] ?? `a deterministic check for ${rule.rule_key}.`
: `/trekreview flags it as ${rule.rule_key}; read the diff for it before the step is called done.`;
const ids = evidence.map((e) => String(e.id ?? '').slice(0, 8)).join(', ');
const ids = evidence.map((e) => clip(String(e.id ?? '').slice(0, 8), 8)).join(', ');
const body = `Mistake: ${clip(rule.description, 160)} Seen: ${seen}. · Fix: ${fix} · How to check: ${how} (evidence: ${ids})`;
return `- [${rule.rule_key}] (${rule.memory_route}) ${body}`;
const line = `- [${rule.rule_key}] (${rule.memory_route}) ${body}`;
if (line.length <= LINE_MAX) return line;
// Shorten the free-text parts, never the three labels.
const short = evidence.map((e) => `"${clip(e.title ?? e.rule_key, 40)}" (${clip(e.file, 40)})`).join(', ');
return `- [${rule.rule_key}] (${rule.memory_route}) Mistake: ${clip(rule.description, 120)} Seen: ${short}. · Fix: ${clip(fix, 120)} · How to check: ${clip(how, 160)} (evidence: ${ids})`;
}
/**
@ -195,7 +211,7 @@ function composeLine(rule, evidence) {
*/
export function validateLessonLine(line) {
if (typeof line !== 'string' || line.trim().length === 0) return { valid: false, reason: 'empty' };
if (line.includes('\n')) return { valid: false, reason: 'multi-line' };
if (/[\u0000-\u001f\u007f\u2028\u2029]/.test(line)) return { valid: false, reason: 'control or line-break character' };
if (line.length > LINE_MAX) return { valid: false, reason: `longer than ${LINE_MAX}` };
if (line.includes('<!--') || line.includes('-->')) return { valid: false, reason: 'marker text' };
const a = line.indexOf('Mistake:');
@ -217,27 +233,41 @@ export function proposeLessons(repo) {
const already = writtenClasses(repo);
const byKey = new Map();
for (const review of reviews) {
const firstPerKey = new Map();
const perKey = new Map();
for (const f of review.findings) {
if (!firstPerKey.has(f.rule_key)) firstPerKey.set(f.rule_key, f);
if (f.original_rule_key !== undefined) continue; // normalised from an unknown key: not a class
if (!perKey.has(f.rule_key)) perKey.set(f.rule_key, []);
perKey.get(f.rule_key).push(f);
}
for (const [key, f] of firstPerKey) {
for (const [key, fs] of perKey) {
if (!byKey.has(key)) byKey.set(key, []);
byKey.get(key).push({ ...f, review: review.identity, source: review.source });
byKey.get(key).push({ review: review.identity, source: review.source, findings: fs });
}
}
const proposals = [];
for (const [key, evidence] of [...byKey].sort(([a], [b]) => a.localeCompare(b))) {
for (const [key, sightings] of [...byKey].sort(([a], [b]) => a.localeCompare(b))) {
// Two reviews, and two DIFFERENT findings: pick, per review, a finding whose
// id has not been used yet. The same unfixed finding twice is one defect.
const usedIds = new Set();
const evidence = [];
for (const s of sightings) {
const f = s.findings.find((x) => !usedIds.has(x.id ?? `${x.file}:${x.line}`));
if (!f) continue;
usedIds.add(f.id ?? `${f.file}:${f.line}`);
evidence.push({ ...f, review: s.review, source: s.source });
}
if (evidence.length < RECURRENCE_THRESHOLD) continue;
const rule = getRule(key);
if (!rule || rule.memory_route === null || rule.memory_route === undefined) continue;
if (already.has(key)) continue;
const ev = evidence.slice(0, RECURRENCE_THRESHOLD);
const line = composeLine(rule, ev);
if (!validateLessonLine(line).valid) continue; // never propose a line apply would refuse
proposals.push({
rule_key: key,
route: rule.memory_route,
target: `${LESSONS_DIR}/${LESSONS_FILE_NAME}`,
line: composeLine(rule, ev),
line,
evidence: evidence.map((e) => ({ id: e.id ?? null, review: e.review, source: e.source, file: e.file ?? null, title: e.title ?? null })),
});
}
@ -247,6 +277,9 @@ export function proposeLessons(repo) {
// ---- writing an approved lesson ---------------------------------------------
function atomicWriteText(path, text) {
// Write THROUGH a symlink (CLAUDE.md -> AGENTS.md is common): rename onto the
// real target, so the link survives and the target gets the change.
if (existsSync(path) && lstatSync(path).isSymbolicLink()) path = realpathSync(path);
mkdirSync(dirname(path), { recursive: true });
const tmp = `${path}.tmp-${process.pid}`;
writeFileSync(tmp, text);
@ -254,7 +287,7 @@ function atomicWriteText(path, text) {
}
function withPointer(claudeText) {
if (claudeText.includes(POINTER_START)) return null;
if (claudeText.includes(POINTER_START) || claudeText.includes(POINTER_END)) return null;
const sep = claudeText.endsWith('\n') ? '\n' : '\n\n';
return `${claudeText}${sep}${POINTER_START}\n${POINTER_LINE}\n${POINTER_END}\n`;
}
@ -285,6 +318,9 @@ export function applyLesson(repo, ruleKey, opts = {}) {
const path = lessonsPath(repo);
const current = existsSync(path) ? readFileSync(path, 'utf-8') : '';
let block = readBlock(current);
if (block && block.damaged) {
return { written: false, reason: 'markers-damaged', detail: `${LESSONS_DIR}/${LESSONS_FILE_NAME} must hold exactly one start marker before one end marker — fix it by hand` };
}
let base = current;
if (!block) {
const sep = current.length === 0 || current.endsWith('\n') ? '' : '\n';

9
lib/review/rule-catalogue.mjs
View file

@ -12,8 +12,9 @@
// review.md files is proposed as project memory (P5, lib/review/recurrence.mjs):
// 'check' - mechanical: a deterministic check can catch it;
// 'rule' - judgement: a line in the target repo's `.claude/rules/` file;
// null - review hygiene (about the review itself, not the target code):
// never proposed.
// null - never proposed: review hygiene (about the review itself, not the
// target code), or a catch-all key (the two engine keys) whose
// members share nothing but their source.
// Routing borrowed from Pocock `skills/in-progress/retro/SKILL.md:19` (MIT).
export const SEVERITY_VALUES = Object.freeze(['BLOCKER', 'MAJOR', 'MINOR', 'SUGGESTION']);
@ -117,14 +118,14 @@ export const RULE_CATALOGUE = Object.freeze([
severity: 'MAJOR',
category: 'correctness',
description: "A defect reported by Claude Code's /code-review that no narrower rule_key covers.",
memory_route: 'rule',
memory_route: null,
}),
Object.freeze({
rule_key: 'SECURITY_REVIEW_VULNERABILITY',
severity: 'BLOCKER',
category: 'security',
description: "A vulnerability reported by Claude Code's /security-review that no narrower rule_key covers.",
memory_route: 'rule',
memory_route: null,
}),
]);

84
lib/review/verdict-gate.mjs
View file

@ -57,28 +57,82 @@ export function decideVerdict(text) {
return { exit: EXIT.BLOCK, verdict, reason: 'verdict BLOCK' };
}
// ALLOW / WARN: the file must back the greener direction.
const block = extractFindingsBlock(r.parsed.body ?? '');
if (block === null) {
// ALLOW / WARN: the file must back the greener direction, from every place a
// BLOCKER can be written. A shape the gate cannot read is exit 2, never 0.
const body = r.parsed.body ?? '';
const fmText = text.slice(0, Math.max(0, text.indexOf('\n---', 3)));
if ((fmText.match(/^verdict\s*:/gm) ?? []).length > 1) {
return undecided('frontmatter carries more than one verdict line', verdict);
}
const lastFence = lastFenceLanguage(body);
if (lastFence === null) {
return undecided(`verdict ${verdict} has no trailing JSON block to back it`, verdict);
}
if (lastFence !== 'json') {
return undecided(`the last fenced block is "${lastFence || '(no language)'}", not the trailing \`json\` block`, verdict);
}
const block = extractFindingsBlock(body);
let payload;
try {
payload = JSON.parse(block);
} catch (e) {
return undecided(`trailing JSON block does not parse: ${e.message}`, verdict);
}
if (payload?.verdict !== undefined && payload.verdict !== verdict) {
const shape = checkPayloadShape(payload);
if (shape) return undecided(`trailing JSON block: ${shape}`, verdict);
if (payload.verdict !== verdict) {
return undecided(`frontmatter verdict ${verdict} disagrees with the JSON block verdict ${payload.verdict}`, verdict);
}
const blockerFindings = Array.isArray(payload?.findings)
? payload.findings.filter((f) => f?.severity === 'BLOCKER').length
: 0;
const blockerCount = Number(payload?.counts?.BLOCKER ?? 0);
if (blockerFindings > 0 || blockerCount > 0) {
return undecided(`verdict ${verdict} over ${Math.max(blockerFindings, blockerCount)} BLOCKER finding(s)`, verdict);
const blockerFindings = payload.findings.filter((f) => String(f.severity).toUpperCase() === 'BLOCKER').length;
const blockerCount = Object.entries(payload.counts)
.filter(([k]) => k.toUpperCase() === 'BLOCKER')
.reduce((n, [, v]) => n + Number(v || 0), 0);
const blockerSection = blockerSectionEntries(body);
if (blockerFindings > 0 || blockerCount > 0 || blockerSection > 0) {
return undecided(`verdict ${verdict} over BLOCKER finding(s) (json: ${blockerFindings}, counts: ${blockerCount}, ## Findings (BLOCKER): ${blockerSection})`, verdict);
}
return { exit: EXIT.PASS, verdict, reason: `verdict ${verdict}` };
const fmIds = new Set(Array.isArray(fm.findings) ? fm.findings : []);
const jsonIds = new Set(payload.findings.map((f) => f.id));
if (fmIds.size !== jsonIds.size || [...fmIds].some((id) => !jsonIds.has(id))) {
return undecided('frontmatter findings: list and the JSON block finding ids differ', verdict);
}
const warnReasons = Array.isArray(payload.allow_blocked_by) ? payload.allow_blocked_by : [];
return { exit: EXIT.PASS, verdict, reason: `verdict ${verdict}`, warn_reasons: warnReasons };
}
const FENCE_RE = /^```[ \t]*([A-Za-z0-9_+-]*)[^\n]*\n[\s\S]*?^```[ \t]*$/gm;
/** Language tag of the LAST fenced block in `body` (lower-cased), '' for none, null when no fence. */
function lastFenceLanguage(body) {
let last = null;
let m;
FENCE_RE.lastIndex = 0;
while ((m = FENCE_RE.exec(body)) !== null) last = m[1].toLowerCase();
return last;
}
/** The JSON block shape the gate needs; returns an error string or null. */
function checkPayloadShape(p) {
if (p === null || typeof p !== 'object' || Array.isArray(p)) return 'not an object';
if (!VERDICT_VALUES.includes(p.verdict)) return `verdict must be one of ${VERDICT_VALUES.join('|')}`;
if (p.counts === null || typeof p.counts !== 'object' || Array.isArray(p.counts)) return 'counts must be an object';
if (!Array.isArray(p.findings)) return 'findings must be an array';
for (const [i, f] of p.findings.entries()) {
if (f === null || typeof f !== 'object' || typeof f.severity !== 'string' || typeof f.id !== 'string') {
return `findings[${i}] needs a string id and severity`;
}
}
return null;
}
/** Number of `### ` entries under `## Findings (BLOCKER)`. */
function blockerSectionEntries(body) {
const start = body.search(/^##\s+Findings\s*\(BLOCKER\)/im);
if (start === -1) return 0;
const rest = body.slice(start).replace(/^[^\n]*\n/, '');
const next = rest.search(/^##\s+/m);
const section = next === -1 ? rest : rest.slice(0, next);
return (section.match(/^###\s+\S/gm) ?? []).length;
}
/**
@ -106,9 +160,11 @@ function isMain() {
if (isMain()) {
const args = process.argv.slice(2);
const filePath = args.find((a) => !a.startsWith('--'));
if (!filePath) {
process.stderr.write('Usage: verdict-gate.mjs [--json] <review.md>\n');
const files = args.filter((a) => !a.startsWith('--'));
const filePath = files[0];
if (files.length !== 1) {
// Exactly one file: a glob would otherwise gate on its first match only.
process.stderr.write('Usage: verdict-gate.mjs [--json] <review.md> (exactly one file)\n');
process.exit(EXIT.UNDECIDED);
}
let result;

12
lib/validators/review-validator.mjs
View file

@ -126,7 +126,10 @@ export function validateReviewContent(text, opts = {}) {
// append to a plan unchanged: `### Step R<n>: …` with `Source finding: <id>`.
// Borrowed: review gaps become appended, numbered tasks with a source-ref,
// never rewriting earlier ones (Spec Kit `templates/commands/converge.md:156-163`, MIT).
const blockerIds = blockerIdsFromBody(body);
// 1.1+ only (N-1 window): a 1.0 review never carried the section.
const v = typeof fm.review_version === 'string' ? fm.review_version.match(/^(\d+)\.(\d+)$/) : null;
const atLeast11 = v && (Number(v[1]) > 1 || (Number(v[1]) === 1 && Number(v[2]) >= 1));
const blockerIds = atLeast11 ? blockerIdsFromBody(body) : [];
if (blockerIds.length > 0) {
const steps = remediationStepSources(body);
for (const id of blockerIds) {
@ -142,6 +145,13 @@ export function validateReviewContent(text, opts = {}) {
}
}
// 1.1+: what the coordinator set aside is listed, never left out (ordre 5).
if (atLeast11 && !/^##\s+Declined to Judge\b/m.test(body)) {
const issueObj = issue('REVIEW_MISSING_SECTION', 'Required body section missing: ## Declined to Judge (review_version ≥ 1.1)');
if (strict) errors.push(issueObj);
else warnings.push(issueObj);
}
for (const section of REVIEW_BODY_SECTIONS) {
const re = new RegExp(`^##\\s+${section}\\b`, 'm');
if (!re.test(body)) {

BIN
scripts/trekreview-armB.workflow.mjs
View file

Binary file not shown.

10
tests/lib/coordinator-contract.test.mjs
View file

@ -157,6 +157,16 @@ test('judgeFilter — only MINOR/SUGGESTION text is trimmed, and every trim is c
assert.ok(j.kept[0].detail.endsWith('[…trimmed]'), 'the cut is visible in the text');
});
test('judgeFilter — a trimmed MINOR keeps its "Also flagged by" provenance', () => {
const [f] = dedupByTriplet([
{ file: 'x.mjs', line: 1, rule_key: 'MISSING_ERROR_HANDLING', severity: 'MINOR', title: 'a', detail: 'q'.repeat(900), reviewer: 'code-correctness-reviewer' },
{ file: 'x.mjs', line: 1, rule_key: 'MISSING_ERROR_HANDLING', severity: 'MINOR', title: 'b', detail: 'r', reviewer: 'claude-code-review' },
]);
const j = judgeFilter([f]);
assert.ok(j.kept[0].detail.length <= 800);
assert.match(j.kept[0].detail, /Also flagged by claude-code-review: b\.$/);
});
test('judgeFilter — the trim decision reads the catalogue tier, not the reviewer label', () => {
// MISSING_TEST is MAJOR in the catalogue; a reviewer calling it MINOR must
// not get its text cut.

34
tests/lib/engine-findings.test.mjs
View file

@ -69,6 +69,40 @@ test('engine-findings — engine and brief control meet in one verdict; engine +
assert.deepEqual([...a.raised_by].sort(), ['brief-conformance-reviewer', 'claude-code-review']);
});
test('engine-findings — a "narrower" key can never lower a security BLOCKER', () => {
const r = toReviewerPayload({ engine: 'security-review', findings: [
{ file: 'a.mjs', line: 1, summary: 's', rule_key: 'MISSING_ERROR_HANDLING' }, // MINOR → refused
{ file: 'b.mjs', line: 1, summary: 's', rule_key: 'SECURITY_INJECTION' }, // BLOCKER → kept
] });
assert.deepEqual(r.payload.findings.map((f) => [f.rule_key, f.severity]),
[['SECURITY_REVIEW_VULNERABILITY', 'BLOCKER'], ['SECURITY_INJECTION', 'BLOCKER']]);
});
test('engine-findings — absolute paths under the repo root match the scope; a numeric-string line is kept', () => {
const r = toReviewerPayload({ engine: 'code-review', findings: [
{ file: '/abs/repo/lib/a.mjs', line: '12', summary: 's' },
] }, { scopeFiles: ['lib/a.mjs'], repoRoot: '/abs/repo' });
assert.equal(r.payload.findings.length, 1);
assert.equal(r.payload.findings[0].file, 'lib/a.mjs');
assert.equal(r.payload.findings[0].line, 12);
});
test('engine-findings — an empty scope list is an error, never "everything out of scope"', () => {
const r = toReviewerPayload({ engine: 'security-review', findings: [{ file: 'a.mjs', summary: 's' }] }, { scopeFiles: [] });
assert.equal(r.ok, false);
});
test('engine-findings — an out-of-scope engine finding forbids ALLOW and is declined to judge', () => {
const r = toReviewerPayload({ engine: 'security-review', findings: [{ file: 'old/x.mjs', line: 2, summary: 'leak' }] },
{ scopeFiles: ['in.mjs'] });
const result = runContract(
[r.payload, { reviewer: 'brief-conformance-reviewer', findings: [] }],
{ expectedReviewers: [r.payload.reviewer, 'brief-conformance-reviewer'], engineOutOfScope: r.out_of_scope },
);
assert.equal(result.verdict, 'WARN');
assert.deepEqual(result.declined_to_judge.map((d) => d.reason), ['engine:out-of-scope']);
});
test('engine-findings — an engine that was expected but never reported forbids ALLOW', () => {
const result = runContract([{ reviewer: 'brief-conformance-reviewer', findings: [] }],
{ expectedReviewers: ['brief-conformance-reviewer', ENGINES['security-review'].reviewer] });

57
tests/lib/review-prose-pins.test.mjs
View file

@ -95,6 +95,63 @@ test('trekreview — a BLOCKER is written as a numbered Step R<n> with Source fi
assert.ok(h6.includes('verdict-gate.mjs'), 'Handover 6 names the gate');
});
test('template — templates/trekreview-template.md validates strictly and the gate reads it as BLOCK', async () => {
const { validateReview } = await import('../../lib/validators/review-validator.mjs');
const { decideVerdictFile } = await import('../../lib/review/verdict-gate.mjs');
const path = join(ROOT, 'templates', 'trekreview-template.md');
const v = validateReview(path, { strict: true });
assert.equal(v.valid, true, JSON.stringify(v.errors));
assert.equal(decideVerdictFile(path).exit, 1);
});
test('trekreview — the zero-diff review the prose prescribes passes the gate as ALLOW', async () => {
const { decideVerdict } = await import('../../lib/review/verdict-gate.mjs');
const p3 = section(CMD, '## Phase 3');
assert.ok(p3.includes('findings: []') && p3.includes('"findings": []'), 'Phase 3 prescribes both');
const text = `---
type: trekreview
review_version: "1.1"
task: "t"
slug: s
project_dir: p/
brief_path: p/brief.md
scope_sha_end: abc
reviewed_files_count: 0
verdict: ALLOW
findings: []
---
## Executive Summary
No diff.
## Coverage
No diff.
## Declined to Judge
None.
Trimmed: 0
## Remediation Summary
No diff.
\`\`\`json
{"verdict": "ALLOW", "counts": {"BLOCKER": 0, "MAJOR": 0, "MINOR": 0, "SUGGESTION": 0}, "findings": []}
\`\`\`
`;
assert.equal(decideVerdict(text).exit, 0, decideVerdict(text).reason);
});
test('trekreview — Phase 8.7 points recurrence at the session directory, not the git toplevel', () => {
const p = section(CMD, '## Phase 8.7');
assert.ok(p.includes('--repo "$PWD"'));
assert.ok(!p.includes('--repo "$(git rev-parse --show-toplevel)"'));
});
// ---- P5: project memory --------------------------------------------------------
test('trekreview — Phase 8.7 asks the operator per line (AskUserQuestion) before recurrence.mjs apply', () => {

72
tests/lib/review-recurrence.test.mjs
View file

@ -8,7 +8,7 @@
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, existsSync } from 'node:fs';
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, existsSync, symlinkSync, lstatSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
@ -224,6 +224,76 @@ test('recurrence — an earlier committed version of the same review.md counts a
} finally { rmSync(repo, { recursive: true, force: true }); }
});
// ---- Independent review of d88245c: each of these was wrong before ----------
test('recurrence — the SAME unfixed finding in two iterations is one defect, not a recurring class', () => {
const repo = repoWith([
{ slug: 'one', sha: 'aaa', findings: [finding('cc', 'MISSING_TEST')] },
{ slug: 'one', sha: 'bbbb', findings: [finding('cc', 'MISSING_TEST')] },
]);
try {
assert.deepEqual(mod.proposeLessons(repo).proposals, []);
} finally { rmSync(repo, { recursive: true, force: true }); }
});
test('recurrence — catch-all engine keys and keys normalised from unknown ones are never proposed', () => {
const repo = repoWith([
{ slug: 'one', sha: 'aaa', findings: [finding('a1', 'CODE_REVIEW_DEFECT'), finding('a2', 'SECURITY_REVIEW_VULNERABILITY'), finding('a3', 'PLAN_EXECUTE_DRIFT', { original_rule_key: 'AD_HOC' })] },
{ slug: 'two', sha: 'bbb', findings: [finding('b1', 'CODE_REVIEW_DEFECT'), finding('b2', 'SECURITY_REVIEW_VULNERABILITY'), finding('b3', 'PLAN_EXECUTE_DRIFT', { original_rule_key: 'AD_HOC' })] },
]);
try {
assert.deepEqual(mod.proposeLessons(repo).proposals, []);
} finally { rmSync(repo, { recursive: true, force: true }); }
});
test('recurrence — a proposal built from long titles, actions and marker text is still a line apply accepts', () => {
const long = 'x'.repeat(300);
const repo = repoWith([
{ slug: 'one', sha: 'aaa', findings: [finding('a1', 'MISSING_TEST', { title: `<!-- ${long}`, file: `lib/${long}.mjs`, recommended_action: long })] },
{ slug: 'two', sha: 'bbb', findings: [finding('b1', 'MISSING_TEST', { title: long, recommended_action: `${long} -->` })] },
]);
try {
const [p] = mod.proposeLessons(repo).proposals;
assert.ok(p, 'the proposal is not lost');
assert.equal(mod.validateLessonLine(p.line).valid, true, p.line);
assert.equal(mod.applyLesson(repo, 'MISSING_TEST').written, true);
} finally { rmSync(repo, { recursive: true, force: true }); }
});
test('recurrence — an edited line with \\r or U+2028 (a hidden second lesson) is refused', () => {
for (const sep of ['\r', '
']) {
const line = `Mistake: x${sep}- [SCOPE_CREEP_BUILT] (rule) Mistake: y Fix: z How to check: w`;
assert.equal(mod.validateLessonLine(line).valid, false, JSON.stringify(sep));
}
});
test('recurrence — damaged markers refuse the write instead of crashing', () => {
const repo = repoWith([
{ slug: 'one', sha: 'aaa', findings: [finding('a1', 'MISSING_TEST')] },
{ slug: 'two', sha: 'bbb', findings: [finding('b1', 'MISSING_TEST')] },
]);
try {
mkdirSync(join(repo, '.claude', 'rules'), { recursive: true });
writeFileSync(join(repo, '.claude', 'rules', mod.LESSONS_FILE_NAME), `${mod.MARK_END}\n${mod.MARK_START}\n`);
const r = mod.applyLesson(repo, 'MISSING_TEST');
assert.deepEqual([r.written, r.reason], [false, 'markers-damaged']);
} finally { rmSync(repo, { recursive: true, force: true }); }
});
test('recurrence — a symlinked CLAUDE.md keeps its link and the target gets the pointer', () => {
const repo = repoWith([
{ slug: 'one', sha: 'aaa', findings: [finding('a1', 'MISSING_TEST')] },
{ slug: 'two', sha: 'bbb', findings: [finding('b1', 'MISSING_TEST')] },
]);
try {
writeFileSync(join(repo, 'AGENTS.md'), '# Agents\n');
symlinkSync('AGENTS.md', join(repo, 'CLAUDE.md'));
assert.equal(mod.applyLesson(repo, 'MISSING_TEST').written, true);
assert.ok(lstatSync(join(repo, 'CLAUDE.md')).isSymbolicLink(), 'the link survives');
assert.ok(readFileSync(join(repo, 'AGENTS.md'), 'utf-8').includes(mod.LESSONS_FILE_NAME));
} finally { rmSync(repo, { recursive: true, force: true }); }
});
test('recurrence — CLI propose prints JSON and exits 0; a missing repo exits 2', () => {
const repo = repoWith([
{ slug: 'one', sha: 'aaa', findings: [finding('a1', 'MISSING_TEST')] },

61
tests/lib/verdict-gate.test.mjs
View file

@ -14,6 +14,7 @@ import { fileURLToPath } from 'node:url';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const GATE = join(ROOT, 'lib', 'review', 'verdict-gate.mjs');
const { decideVerdict } = existsSync(GATE) ? await import('../../lib/review/verdict-gate.mjs') : {};
function review({ verdict, jsonVerdict, severities = [] }) {
const ids = severities.map((_, i) => String(i + 1).padStart(40, 'a'));
@ -135,6 +136,66 @@ test('verdict-gate — --json prints the decision as JSON', () => {
}
});
// ---- Adversarial inputs (independent review of d88245c): each was exit 0 ----
function swapJson(text, jsonSrc) {
return text.replace(/```json\n[\s\S]*?```\n$/, `\`\`\`json\n${jsonSrc}\n\`\`\`\n`);
}
test('verdict-gate — a greener verdict over a malformed or evasive JSON block is exit 2', () => {
const base = review({ verdict: 'WARN', severities: ['MAJOR'] });
assert.equal(decideVerdict(base).exit, 0, 'control: the untouched WARN review passes');
const id = String(1).padStart(40, 'a');
for (const json of [
'{}',
'null',
`{"verdict":"WARN","counts":{"BLOCKER":0},"findings":[{"id":"${id}","severity":"blocker"}]}`,
`{"verdict":"WARN","counts":{"blocker":1},"findings":[{"id":"${id}","severity":"MAJOR"}]}`,
`{"verdict":"WARN","counts":{},"findings":{"${id}":{"severity":"BLOCKER"}}}`,
]) {
assert.equal(decideVerdict(swapJson(base, json)).exit, 2, json);
}
});
test('verdict-gate — the trailing json block must be the last fence', () => {
const base = review({ verdict: 'WARN', severities: ['MAJOR'] });
assert.equal(decideVerdict(`${base}\n\`\`\`json\n{"verdict":"WARN","counts":{},"findings":[]}\n\`\`\`\n`).exit, 2,
'a later json fence cannot replace the real block');
assert.equal(decideVerdict(base.replace(/```json\n(?=[\s\S]*```\n$)/, '```jsonc\n')).exit, 2,
'a jsonc last fence is not the json block');
});
test('verdict-gate — a ## Findings (BLOCKER) entry under a WARN verdict is exit 2', () => {
const base = review({ verdict: 'WARN', severities: ['MAJOR'] });
const t = base.replace('## Coverage', '## Findings (BLOCKER)\n\n### deadbeef\n\n- file: a.mjs\n\n## Coverage');
assert.equal(decideVerdict(t).exit, 2);
});
test('verdict-gate — two verdict lines in the frontmatter are exit 2', () => {
const base = review({ verdict: 'ALLOW', severities: ['MINOR'] });
assert.equal(decideVerdict(base.replace('verdict: ALLOW\n', 'verdict: BLOCK\nverdict: ALLOW\n')).exit, 2);
});
test('verdict-gate — frontmatter finding ids that differ from the JSON block are exit 2', () => {
const base = review({ verdict: 'WARN', severities: ['MAJOR'] });
const t = base.replace(` - ${String(1).padStart(40, 'a')}`, ` - ${'b'.repeat(40)}`);
assert.equal(decideVerdict(t).exit, 2);
});
test('verdict-gate — more than one file argument is refused (exit 2), never a first-match decision', () => {
const dir = mkdtempSync(join(tmpdir(), 'verdict-gate-'));
try {
const a = join(dir, 'a.md');
const b = join(dir, 'b.md');
writeFileSync(a, review({ verdict: 'ALLOW', severities: ['MINOR'] }));
writeFileSync(b, review({ verdict: 'BLOCK', severities: ['BLOCKER'] }));
const r = spawnSync(process.execPath, [GATE, a, b], { encoding: 'utf-8' });
assert.equal(r.status, 2);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
test('verdict-gate — run through a symlink and a path with a space it still decides (realpath main guard)', () => {
// The old `file://${argv[1]}` guard is a no-op under a symlink or a space:
// the CLI would exit 0 without reading anything — a silent green.

71
tests/scripts/trekreview-armb-workflow.test.mjs Normal file
View file

@ -0,0 +1,71 @@
// tests/scripts/trekreview-armb-workflow.test.mjs
// The opt-in Workflow substrate for /trekreview Phase 5–6, run for real with
// stubbed Workflow globals (agent/parallel/phase/log). Ordre 5: a reviewer or
// engine that did not report forbids ALLOW, and the verdict's floor is
// recomputed from the findings the coordinator returns.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const SRC = readFileSync(join(ROOT, 'scripts', 'trekreview-armB.workflow.mjs'), 'utf-8')
.replace('export const meta', 'const meta');
const AsyncFunction = Object.getPrototypeOf(async () => {}).constructor;
const run = new AsyncFunction('args', 'agent', 'parallel', 'phase', 'log', SRC);
const BASE = { briefPath: 'b.md', diffPath: 'd.diff', triage: '' };
const engine = (reviewer, findings = []) => ({ reviewer, findings });
function stubs({ reviewers = [], coordinator = null } = {}) {
let i = 0;
return {
agent: async (_prompt, opts) => (opts.label === 'coordinator' ? coordinator : reviewers[i++] ?? null),
parallel: async (thunks) => Promise.all(thunks.map((t) => t())),
phase: () => {},
log: () => {},
};
}
async function exec(args, s) {
return run(args, s.agent, s.parallel, s.phase, s.log);
}
test('armB — both reviewers and both engines clean → ALLOW (known-positive control)', async () => {
const s = stubs({ reviewers: [{ reviewer: 'c', findings: [] }, { reviewer: 'k', findings: [] }] });
const r = await exec({ ...BASE, enginePayloads: [engine('claude-code-review'), engine('claude-security-review')] }, s);
assert.equal(r.verdict, 'ALLOW');
assert.deepEqual(r.missing_engines, []);
});
test('armB — engines the caller never mentioned are missing → not ALLOW', async () => {
const s = stubs({ reviewers: [{ reviewer: 'c', findings: [] }, { reviewer: 'k', findings: [] }] });
const r = await exec({ ...BASE }, s);
assert.equal(r.verdict, 'WARN');
assert.deepEqual(r.missing_engines, ['claude-code-review', 'claude-security-review']);
});
test('armB — a reviewer that returned nothing → not ALLOW', async () => {
const s = stubs({ reviewers: [null, { reviewer: 'k', findings: [] }] });
const r = await exec({ ...BASE, enginePayloads: [engine('claude-code-review'), engine('claude-security-review')] }, s);
assert.equal(r.verdict, 'WARN');
});
test('armB — a coordinator ALLOW over a BLOCKER finding is recomputed to BLOCK', async () => {
const f = { severity: 'BLOCKER', rule_key: 'UNIMPLEMENTED_CRITERION', file: 'a.mjs', line: 1, title: 't', detail: 'd' };
const s = stubs({
reviewers: [{ reviewer: 'c', findings: [f] }, { reviewer: 'k', findings: [] }],
coordinator: { verdict: 'ALLOW', findings: [f] },
});
const r = await exec({ ...BASE, enginePayloads: [engine('claude-code-review'), engine('claude-security-review')] }, s);
assert.equal(r.verdict, 'BLOCK');
});
test('armB — quick mode expects only the /code-review engine', async () => {
const s = stubs({ reviewers: [{ reviewer: 'k', findings: [] }] });
const r = await exec({ ...BASE, quick: true, enginePayloads: [engine('claude-code-review')] }, s);
assert.equal(r.verdict, 'ALLOW');
assert.deepEqual(r.missing_engines, []);
});

19
tests/validators/review-validator.test.mjs
View file

@ -150,7 +150,9 @@ test('validateReview — a verdict outside ALLOW|WARN|BLOCK is an error in both
// ---- A BLOCKER is written as a plan step Handover 6 can append (ordre 5) ----
const BLOCKER_ID = '763d174e6c519fafbadcba5d1706708479e36e61';
const WITH_BLOCKER = GOOD_REVIEW.replace('verdict: ALLOW', 'verdict: BLOCK') + `
const WITH_BLOCKER = GOOD_REVIEW.replace('verdict: ALLOW', 'verdict: BLOCK')
.replace('review_version: "1.0"', 'review_version: "1.1"')
.replace('## Remediation Summary', '## Declined to Judge\n\nNone.\n\nTrimmed: 0\n\n## Remediation Summary') + `
\`\`\`json
{"verdict": "BLOCK", "counts": {"BLOCKER": 1, "MAJOR": 0, "MINOR": 0, "SUGGESTION": 0},
"findings": [{"id": "${BLOCKER_ID}", "severity": "BLOCKER", "rule_key": "UNIMPLEMENTED_CRITERION", "file": "lib/login.mjs", "line": 23}]}
@ -178,6 +180,11 @@ test('validateReview — strict: a BLOCKER with its "Step R1 … Source finding"
assert.equal(r.valid, true, JSON.stringify(r.errors));
});
test('validateReview — a 1.0 review with a BLOCKER is not held to the 1.1 remediation-step rule (N-1)', () => {
const v10 = WITH_BLOCKER.replace('review_version: "1.1"', 'review_version: "1.0"');
assert.equal(validateReviewContent(v10, { strict: true }).valid, true);
});
test('validateReview — soft: a missing remediation step only warns', () => {
const r = validateReviewContent(WITH_BLOCKER, { strict: false });
assert.equal(r.valid, true);
@ -191,6 +198,16 @@ test('validateReview — a clean review with zero findings validates with `findi
assert.equal(validateReviewContent(GOOD_REVIEW.replace(ids, 'findings:\n'), { strict: true }).valid, false);
});
test('validateReview — strict: a 1.1 review needs ## Declined to Judge; a 1.0 review does not', () => {
const v11 = GOOD_REVIEW.replace('review_version: "1.0"', 'review_version: "1.1"');
const r = validateReviewContent(v11, { strict: true });
assert.equal(r.valid, false);
assert.ok(r.errors.find(e => e.code === 'REVIEW_MISSING_SECTION' && /Declined to Judge/.test(e.message)));
const fixed = v11.replace('## Remediation Summary', '## Declined to Judge\n\nNone.\n\nTrimmed: 0\n\n## Remediation Summary');
assert.equal(validateReviewContent(fixed, { strict: true }).valid, true);
assert.equal(validateReviewContent(GOOD_REVIEW, { strict: true }).valid, true, '1.0 stays valid');
});
test('review-validator CLI — run through a symlink and a path with a space it still validates (exit 1, not a silent 0)', () => {
const dir = mkdtempSync(join(tmpdir(), 'review validator '));
try {