open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

fix(review): convert refuses a partial or scoped Claude Security report; the stamp check is a trace (decisions 223, 224)

Browse source 
Two fail-closed checks in security-engine.mjs convert (decision 224), both
"not measured" (exit 2, the reason on stderr, nothing on stdout):
- the stamp's findings.total is not a non-negative integer, or the JSONL
  holds another number of records (a JSONL that lost the only BLOCKER line
  converted clean before; part 3 verifier, F3);
- the stamp's scope is not an empty list: a scan limited by --scope, an
  absent scope or a non-list scope (Claude Security 0.12.0 writes scope: []
  for an unscoped scan; F4).
The real Claude Security 0.12.0 probe report still converts: exit 0, one
BLOCKER, two below_threshold.

Decision 223, text only: the module header and SECURITY.md keep the
head/base check and say it is a trace, not a signature: code the reviewed
run executed can write a report folder of its own (convert reads the
matching one whose name sorts last) or delete the real one first (F2, a
known issue). /trekreview's not-measured list names the two new reasons.

The test helper writes the real stamp shape by default (findings.total =
the records in the JSONL written, scope: []). New tests for behaviour that
had none (F6): no fallback from an unverified newest match, a two-stamp
folder passed over, a symlinked stamp, JSONL or folder refused, a malformed
or absent scan_prefix, an unrecorded empty_diff or empty_scope.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-11-08 18:24:39 +01:00
commit f6e8529bdd
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
5 changed files with 179 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

14
tests/helpers/claude-security-report.mjs
View file

@ -12,6 +12,11 @@
//
// `writeReport` takes the commit range and lets a test change any part of the
// stamp (`stamp: (s) => ...`) or write the JSONL text itself (`jsonl`).
// By default the stamp has the real shape of an unscoped changes scan:
// `findings.total` is the number of records in the JSONL written (the
// non-empty lines of `jsonl` when a test gives it; render_report.py writes
// `len(findings)`), and `scope` is `[]` (write_scan_meta.py: no `--scope`).
// A test changes either through `stamp`.
import { mkdirSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
@ -42,8 +47,8 @@ export function record(over = {}) {
};
}
/** The stamp of a verified changes scan of `base..head`. */
export function stampOf({ head, base, dirty = false, records = [] }) {
/** The stamp of a verified, unscoped changes scan of `base..head` whose JSONL holds `total` records. */
export function stampOf({ head, base, dirty = false, records = [], total = records.length }) {
const count = (sev) => records.filter((r) => r.severity === sev).length;
return {
generated_at: '2026-11-08T12:00:00+00:00',
@ -61,7 +66,7 @@ export function stampOf({ head, base, dirty = false, records = [] }) {
diff_files: 2, diff_lines: 30, pre_existing: 0, scope_files: 2,
empty_diff: false, empty_scope: false, researchers_dispatched: 3, verification_runs: 1,
},
findings: { total: records.length, critical: count('CRITICAL'), high: count('HIGH'), medium: count('MEDIUM'), low: count('LOW') },
findings: { total, critical: count('CRITICAL'), high: count('HIGH'), medium: count('MEDIUM'), low: count('LOW') },
verification: {
status: 'verified', candidates: records.length, candidates_deduped: records.length,
panel_votes: 3 * records.length, panel_reviewed_findings: records.length,
@ -89,7 +94,8 @@ export function writeReport(dir, { head, base, ts = '20261108-120000', dirty = f
writeFileSync(join(folder, 'CLAUDE-SECURITY-RESULTS.jsonl'),
jsonl ?? records.map((r) => `${JSON.stringify(r)}\n`).join(''));
}
const total = typeof jsonl === 'string' ? jsonl.split('\n').filter((l) => l.trim() !== '').length : records.length;
const name = stampName ?? `CLAUDE-SECURITY-REVISION-${head.slice(0, 12)}${dirty ? '-dirty' : ''}.json`;
writeFileSync(join(folder, name), JSON.stringify(stamp(stampOf({ head, base, dirty, records })), null, 2));
writeFileSync(join(folder, name), JSON.stringify(stamp(stampOf({ head, base, dirty, records, total })), null, 2));
return folder;
}