open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

feat(review): the review is a gate - verdict in the file, CC engine, project memory

Browse source 
Ordre 5 (wave 1, P11 + P5), review_version 1.1.

- K7: `verdict: ALLOW | WARN | BLOCK` is required in review.md (strict
  error, soft warning for 1.0 reviews). New CLI lib/review/verdict-gate.mjs:
  0 = ALLOW/WARN, 1 = BLOCK, 2 = could not decide (never green); a verdict
  greener than the file's own BLOCKERs or JSON block is exit 2. Realpath
  main guard (the old file:// guard exits 0 under a symlink - measured).
- No finding is struck for length: succinctness trims only MINOR/SUGGESTION
  text (catalogue tier), keeps BLOCKER/MAJOR whole, counts trims; the
  coordinator lists what it declined to judge.
- Claude Code's /code-review and /security-review run as the engine via the
  Skill tool (never --fix/--comment/--post/ultra); lib/review/engine-findings.mjs
  turns their findings into a reviewer payload; two catalogue keys
  (CODE_REVIEW_DEFECT, SECURITY_REVIEW_VULNERABILITY), catalogue 12 -> 14.
  An engine that did not run forbids ALLOW.
- Every BLOCKER is written as a numbered `### Step R<n>` with its source
  finding (strict error REVIEW_MISSING_REMEDIATION_STEP).
- P5: lib/review/recurrence.mjs - a finding class in two review.md files
  (current + git history, no new store) becomes one proposal, routed by the
  catalogue's memory_route; written only between Voyage markers in
  .claude/rules/voyage-review-lessons.md under a size budget, CLAUDE.md gets
  a pointer only; the operator approves every line (Phase 8.7).
- Workflow substrate: catalogue enum in step; a missing reviewer or engine
  forbids ALLOW there too (it returned ALLOW on zero findings before).
- Fixtures review-run-A/B said WARN over a BLOCKER; now BLOCK.

Borrowed and credited in code: Superpowers (declined-to-judge), ai-native-sdlc
(cap only nits; mistake/fix/check), Spec Kit (managed block; converge steps),
Pocock (retro routing), Kiro (verdict file + gate; evidence per lesson),
Cherny (engine -> brief control order; review -> memory; budget).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-06 19:45:52 +02:00
commit d88245c06e
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
28 changed files with 2052 additions and 110 deletions
Download patch file Download diff file Expand all files Collapse all files

90
tests/lib/coordinator-contract.test.mjs
View file

@ -127,20 +127,43 @@ test('reasonablenessFilter — citation-less is REFUTED, unknown rule_key is UNV
// ---- Pass 2 — judge --------------------------------------------------------
test('judgeFilter — over-long title and empty recommended_action are UNVERIFIED, not dropped', () => {
// Contract change (fail-closed): both implemented Pass 2 tests read a
// `.length` and never examine the claim, so neither refutes the finding.
test('judgeFilter — an over-long MAJOR title is kept whole; an empty recommended_action is UNVERIFIED', () => {
// Length never removes a finding (ordre 5, P11). Actionability reads a
// `.length` and never examines the claim, so it cannot refute the finding.
// `dropped` is empty here on purpose — the refuting Pass 2 filter (Accuracy)
// is the one this deterministic subset excludes.
const longTitle = 'x'.repeat(101);
const j = judgeFilter([
{ file: 'x.mjs', line: 1, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: 'x'.repeat(101) }, // too long → unverified
{ file: 'x.mjs', line: 1, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: longTitle }, // MAJOR → kept whole
{ file: 'x.mjs', line: 2, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: 'ok', recommended_action: ' ' }, // empty action → unverified
{ file: 'x.mjs', line: 3, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: 'ok' }, // keep (no action field is fine)
]);
assert.equal(j.kept.length, 1);
assert.equal(j.kept.length, 2);
assert.equal(j.kept[0].title, longTitle, 'a MAJOR keeps every character');
assert.equal(j.dropped.length, 0);
assert.equal(j.unverified.length, 2);
assert.deepEqual(j.unverified.map((f) => f.suppressed_reason), ['succinctness:title', 'actionability:empty']);
assert.equal(j.trimmed.length, 0);
assert.deepEqual(j.unverified.map((f) => f.suppressed_reason), ['actionability:empty']);
});
test('judgeFilter — only MINOR/SUGGESTION text is trimmed, and every trim is counted', () => {
const j = judgeFilter([
{ file: 'x.mjs', line: 1, rule_key: 'MISSING_ERROR_HANDLING', severity: 'MINOR', title: 't'.repeat(150), detail: 'd'.repeat(1200) },
{ file: 'x.mjs', line: 2, rule_key: 'MISSING_ERROR_HANDLING', severity: 'MINOR', title: 'short', detail: 'short' },
]);
assert.equal(j.kept.length, 2, 'a trimmed finding is still kept');
assert.equal(j.trimmed.length, 1);
assert.ok(j.kept[0].title.length <= 100 && j.kept[0].detail.length <= 800);
assert.deepEqual(j.kept[0].trimmed_fields, ['title', 'detail']);
assert.ok(j.kept[0].detail.endsWith('[…trimmed]'), 'the cut is visible in the text');
});
test('judgeFilter — the trim decision reads the catalogue tier, not the reviewer label', () => {
// MISSING_TEST is MAJOR in the catalogue; a reviewer calling it MINOR must
// not get its text cut.
const detail = 'd'.repeat(900);
const j = judgeFilter([{ file: 'x.mjs', line: 1, rule_key: 'MISSING_TEST', severity: 'MINOR', title: 't', detail }]);
assert.equal(j.trimmed.length, 0);
assert.equal(j.kept[0].detail, detail);
});
// ---- ingest ----------------------------------------------------------------
@ -229,19 +252,34 @@ test('computeVerdict — a reviewer that did not report forbids ALLOW', () => {
assert.ok(r.allow_blocked_by.some((x) => x.includes('brief-conformance-reviewer')));
});
test('runContract — a BLOCKER dropped for an over-long title cannot yield ALLOW', () => {
// Pass 2 succinctness reads `.length`. It never examines the claim, so it
// cannot establish the finding is unreal -- it is unverified, not refuted.
test('runContract — a BLOCKER with an over-long title is kept and yields BLOCK', () => {
// Before ordre 5 this finding went to `unverified` on `succinctness:title`
// (the fail-closed floor kept it from ALLOW, but it fell out of the list).
const result = runContract([
{ reviewer: 'code-correctness-reviewer', findings: [
{ file: 'lib/auth/jwt.mjs', line: 19, rule_key: 'SECURITY_INJECTION', severity: 'BLOCKER', title: 'x'.repeat(101), detail: 'algo taken from the JWT header' },
] },
]);
assert.notEqual(result.verdict, 'ALLOW', 'an unsubstantiated BLOCKER must never clear the review');
assert.equal(result.findings.length, 0, 'it is still not a kept finding');
assert.equal(result.unverified.length, 1);
assert.equal(result.unverified[0].suppressed_reason, 'succinctness:title');
assert.equal(result.suppressed.length, 1, 'suppressed stays the union of dropped + unverified');
assert.equal(result.verdict, 'BLOCK');
assert.equal(result.findings.length, 1);
assert.equal(result.unverified.length, 0);
assert.equal(result.suppressed.length, 0);
});
test('runContract — declined_to_judge lists every unverified finding with its reason; trimmed_count counts trims', () => {
const result = runContract([
{ reviewer: 'code-correctness-reviewer', findings: [
{ file: 'a.mjs', line: 1, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: 'empty action', recommended_action: '' },
{ file: 'b.mjs', line: 2, rule_key: 'MISSING_ERROR_HANDLING', severity: 'MINOR', title: 'long', detail: 'z'.repeat(850), recommended_action: 'Guard it.' },
] },
]);
assert.equal(result.declined_to_judge.length, 1);
assert.deepEqual(
{ file: result.declined_to_judge[0].file, reason: result.declined_to_judge[0].reason },
{ file: 'a.mjs', reason: 'actionability:empty' },
);
assert.equal(result.trimmed_count, 1);
assert.equal(result.counts.MINOR, 1, 'the trimmed MINOR still counts in its tier');
});
test('runContract — a schema-invalid payload cannot yield ALLOW (an unread reviewer is an absent one)', () => {
@ -333,3 +371,25 @@ test('runContract — an anonymous invalid payload forbids ALLOW on its own, wit
assert.equal(result.unattributable_payloads, 1);
assert.notEqual(result.verdict, 'ALLOW');
});
// ---- No finding is struck for length (ordre 5, P11) --------------------------
//
// The defect (STATE, Åpne funn; S118): a finding whose `detail` ran past 800
// characters went to `unverified` on `succinctness:detail` alone, and S118's most
// serious finding fell out of the Handover 6 list that way. Length is packaging,
// not a claim: only MINOR/SUGGESTION text is trimmed, and every trim is counted.
test('runContract — a BLOCKER with a 900-character detail survives and yields BLOCK', () => {
const detail = 'The refresh token is accepted after logout because the revocation list is never consulted. '.repeat(10).slice(0, 900);
assert.equal(detail.length, 900);
const result = runContract([
{ reviewer: 'code-correctness-reviewer', findings: [
{ file: 'lib/auth/refresh.mjs', line: 31, rule_key: 'SECURITY_INJECTION', severity: 'BLOCKER',
title: 'Revoked refresh token still accepted', detail, recommended_action: 'Check the revocation list before issuing.' },
] },
]);
assert.equal(result.verdict, 'BLOCK');
assert.equal(result.findings.length, 1, 'the long BLOCKER is a kept finding');
assert.equal(result.findings[0].detail, detail, 'its detail is not cut');
assert.equal(result.unverified.length, 0);
});