feat(review): the brief's threat model on the review side (decision 164, 167, 218)

The code-correctness reviewer's dimension 6 shrinks to what the engines do
not read: command, skill and agent markdown carrying shell, with Claude
Code's threshold (a concrete attack path, high confidence), the keys
SECURITY_INJECTION or SECURITY_RISK, and threat_model_ref quoting the
brief's Outside: line; the code-level bullets go to the engines.

The coordinator puts a Voyage security finding with a threat_model_ref
into the trailing JSON block's known_outside, never into findings.
review-finalize gains the pass `outside`, right after engine: it only
raises. An entry the brief at the review's brief_path does not back now
(line gone, threat model not usable, an engine finding, not a security
finding) goes back into the findings at its catalogue tier with a Source
line, R-step and raised verdict, as the engine put-back does; the rest are
listed under the new `## Outside the Threat Model` section.

verdict-gate decideOpenFindings (and so scripts/release-check.mjs)
re-checks every entry against brief_path through the new
notBackedOutside and exits 2 for one not backed. review-validator
shape-checks the optional known_outside (REVIEW_KNOWN_OUTSIDE_MALFORMED).
threat-model.mjs gains reviewThreatModel and isKnownEntry for the three.

/trekreview Phase 8.8 lists the known issues outside the threat model
(not answered, not counted); the review template shows the section and
field; Handover 1 names the optional ## Threat Model and its code,
Handover 6 names known_outside and its code.

Part 2, step 2.5 of the E plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-11-08 09:14:47 +01:00
commit c47061191c
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
10 changed files with 585 additions and 30 deletions

View file

@ -34,7 +34,7 @@ You will receive a prompt containing:
pass — declared signatures, exports, top-level wiring — but no deep
semantic analysis.
- **Brief path** (optional) — `{project_dir}/brief.md`. Read for `brief_ref`
context only. The brief is not your contract — it is the conformance
context and for its `## Threat Model`, when it has one (dimension 6), only. The brief is not your contract — it is the conformance
reviewer's contract. You evaluate code correctness regardless of
what the brief promised.
- **Rule catalogue** — the 18-key catalogue in
@ -118,21 +118,30 @@ literals or example fenced blocks):
Cite the exact line. Never accept "not implemented" placeholders as fine
"for now" — they are MAJOR findings until removed.
### 6. Security surface — `SECURITY_INJECTION` (BLOCKER)
### 6. Security surface (command texts) — `SECURITY_INJECTION` / `SECURITY_RISK` (BLOCKER)
- Untrusted input is interpolated into a shell command (`exec`, `spawn`
with `shell: true`, template-literal command construction).
- Untrusted input is interpolated into a SQL query, an HTML template,
or a regex without escaping.
- File paths are constructed from untrusted input without
`path.normalize` + a base-dir containment check (path traversal).
- A new HTTP endpoint accepts user input and renders it back without
output encoding (XSS).
- Process env vars containing secrets are echoed in logs.
The security engine reads the code; this dimension covers only what the
engines do not read (decision 167): the command, skill and agent markdown in
the diff that carries shell (a bash block or a command line in
`commands/*.md`, `skills/**/*.md`, `agents/*.md`) and that a session runs as
written.
Cite the line and explain the injection vector. Never assume something
is safe because "the input is internal" — that's how supply-chain
attacks become RCE.
- An untrusted value (an argument, a file name, a field a reviewer or an
engine wrote, fetched text) is spliced into a shell line unquoted, or into
a heredoc the shell expands.
- A path is built from such a value with no containment check.
- A secret is echoed into a log, a transcript or a committed file.
Raise one only with a concrete attack path (who sends what, through which
step, to what effect) and high confidence, Claude Code's own threshold for a
security finding; quote the line. Use `SECURITY_INJECTION`, or `SECURITY_RISK`
when no narrower key fits; the engines' key is never yours. Never assume
something is safe because "the input is internal". When the brief has a
`## Threat Model` and the attack path is one an `Outside:` line of it leaves
out, put that line's text (after `Outside: `) in `threat_model_ref`, exactly
as the brief writes it: the coordinator then lists the finding as a known
issue outside the threat model instead of counting it, while the brief's
intent approval is current. Otherwise leave `threat_model_ref` out.
### 7. Hidden dependencies — `UNDECLARED_DEPENDENCY` (MAJOR)
@ -176,6 +185,7 @@ Severity is fixed by `rule_key`. Do NOT override the catalogue:
| `MISSING_TEST` | MAJOR |
| `PLACEHOLDER_IN_CODE` | MAJOR |
| `SECURITY_INJECTION` | BLOCKER |
| `SECURITY_RISK` | BLOCKER |
| `UNDECLARED_DEPENDENCY` | MAJOR |
| `COVERAGE_SILENT_SKIP` | MAJOR |
| `CLAUDE_MD_DEVIATION` | MINOR |
@ -206,7 +216,7 @@ parsers find it by reading the last `json` code fence.
| Cross-file regressions | PLAN_EXECUTE_DRIFT | {N} |
| Test coverage gaps | MISSING_TEST | {N} |
| Placeholder code | PLACEHOLDER_IN_CODE | {N} |
| Security surface | SECURITY_INJECTION | {N} |
| Security surface (command texts) | SECURITY_INJECTION / SECURITY_RISK | {N} |
| Hidden dependencies | UNDECLARED_DEPENDENCY | {N} |
| Instruction files | CLAUDE_MD_DEVIATION / CLAUDE_MD_STALE | {N} |
@ -272,6 +282,8 @@ parsers find it by reading the last `json` code fence.
- `rule_key` MUST be in the catalogue. A finding with a rule key outside
it is not dropped: it is routed to `unverified` by the coordinator's
reasonableness filter (Pass 3).
- `threat_model_ref` (optional, dimension 6 only): the text of the brief's
`Outside:` line the finding falls under, exactly; omit it otherwise.
## Rules

View file

@ -229,9 +229,22 @@ vocabulary on purpose.
is reviewer-asserted and was never substantiated; counting it would let an
unexamined finding *raise* the verdict, which is invention.
**Known issues outside the threat model** (E, decisions 164, 218). A finding
from a Voyage reviewer (no `engine`) under a security key (`SECURITY_INJECTION`,
`SECURITY_RISK`) that carries `threat_model_ref` goes into the JSON block's
`known_outside`, as the reviewer gave it with its `threat_model_ref`, never into
`findings`: no `@@ID_<n>@@`, no `### ` entry, not counted, no effect on the
verdict. You do not judge whether the brief backs it:
`lib/review/review-finalize.mjs` checks each entry against the brief's
`## Threat Model`, puts back into the findings any the brief does not back and
lists the rest under `## Outside the Threat Model`, and the verdict gate checks
again. An engine finding never goes there, with or without the field: it
blocks as before.
### Pass 4 — Compute verdict
Count findings by severity AFTER dedup and filtering. Verdict thresholds:
Count findings by severity AFTER dedup and filtering (a `known_outside` entry
is no finding and is not counted). Verdict thresholds:
| Counts | Verdict |
|--------|---------|
@ -364,6 +377,7 @@ The LAST fenced block in the file is a `json` block:
"counts": { "BLOCKER": N, "MAJOR": N, "MINOR": N, "SUGGESTION": N },
"allow_blocked_by": ["unverified:actionability:empty (1)", "missing-reviewer:claude-security-review", "unattributable-payload (1)"],
"declined_to_judge": [{ "file": "lib/x.mjs", "line": 3, "rule_key": "MISSING_TEST", "reason": "actionability:empty" }],
"known_outside": [{ "severity": "BLOCKER", "rule_key": "SECURITY_RISK", "file": "commands/x.md", "line": 12, "brief_ref": "Threat Model", "title": "...", "detail": "...", "recommended_action": "...", "threat_model_ref": "<the text of the brief's Outside: line>" }],
"trimmed": 0,
"findings": [
{

View file

@ -855,6 +855,16 @@ its reason in one line.
Then run the Phase 8 gate line again and report its result as `gate_exit`.
**Known issues outside the threat model** (E, decisions 164, 218). List each
entry of the review's `## Outside the Threat Model` (the trailing JSON block's
`known_outside`) for the operator in plain words, with the brief's `Outside:`
line it falls under: Voyage's own security findings the brief's threat model
puts outside. They are not answered and do not count. The verdict gate checks
each against the brief on every run: one whose line is gone, or whose intent
approval went stale, makes the gate exit 2 (not measured) until the review
runs again. To lift one back in, the operator changes that `Outside:` line and
approves the brief's intent again.
Run on its own, outside `/trekexecute`, "fixed" leaves the finding open until a
later `/trekreview` round marks it resolved.

View file

@ -88,6 +88,8 @@ An older brief's `phase_signals` key is ignored.
Optional but standard sections: `## Non-Goals`, `## Constraints`, `## Preferences`, `## Non-Functional Requirements`, `## Research Plan`.
**Optional `## Threat Model`** (E, decisions 164, 218; additive, no `brief_version` bump, the `ceremony` / `artifact_store` precedent). One line per bullet, each `- Inside: <text>` (what the work must defend against) or `- Outside: <text>` (what it deliberately does not); a line that is not a bullet is prose. Present and malformed (no such bullet, another label or form, empty text, two such sections) is `BRIEF_THREAT_MODEL_MALFORMED`; absent means every security finding counts, as before. The intent hash covers the section when it is present (`intent-approval.mjs`), so editing, adding or removing it makes the approval stale; a brief without it hashes as before. What consumers do with it: a security finding of Voyage's own reviewers (plan-critic, scope-guardian, the code reviewer's part for command texts; never an engine's) that quotes an `Outside:` line's text exactly in `threat_model_ref` is listed as a known issue instead of counted (`known_outside`, Handover 4 and 6), and only while the intent approval is current by the marker and the approval log; `lib/review/threat-model.mjs` is the one reader.
**Validation strategy:**
| Layer | When | What |
@ -109,7 +111,7 @@ Optional but standard sections: `## Non-Goals`, `## Constraints`, `## Preference
**Pre-2.2 briefs receive zero framing enforcement (producer-elective defense).** The framing-alignment defense is gated at `brief_version ≥ 2.2`, so a brief declaring `2.0` or `2.1` gets **none** of it: `framing` is optional and a missing value is not an error, the `## TL;DR` section is not required, and the `brief-reviewer` memory-alignment dimension scores N/A. The backward-compatibility this buys is deliberate, but it also means the framing defense is **producer-elective**: any producer can sidestep the entire defense — deliberately or by shipping a legacy template — simply by declaring `brief_version: "2.1"`, and the pipeline raises no signal by default. This is a documented property of the contract, not a bug. Two remedies close it for a given consumer: (1) require upstream producers to emit `brief_version: "2.2"` (which forces `framing` + `## TL;DR`); or (2) pass the opt-in **`--min-brief-version <ver>`** flag to `/trekplan` or `/trekresearch` (forwarded to the validator as `--min-version`), which raises a `BRIEF_VERSION_BELOW_MINIMUM` **warning** — never a block — when a brief declares a version below the floor. The flag defaults to off, so the N-1 compatibility window for `2.0`/`2.1` briefs is preserved unless a consumer explicitly opts into the stricter floor.
**Intent approval gate (veikart steg 1) — additive schema, consumer-side requirement.** The two `intent_approved_*` fields are additive and optional in the schema: `brief-validator.mjs` does not require them, `/trekresearch` does not read them, and no `brief_version` bump is involved. The requirement lives in `/trekplan` Phase 1 (`### Read the brief`), which runs `intent-approval.mjs --check` and **halts** on `BRIEF_INTENT_NOT_APPROVED` (no marker), `BRIEF_INTENT_APPROVAL_STALE` (`## Intent` or `## Goal` changed after approval) or `BRIEF_INTENT_APPROVAL_INVALID` (malformed marker, or a section missing, empty or present more than once; a `## ` line inside a code block or HTML comment is not a heading). It is version-independent, so a brief from ANY producer, of any `brief_version`, stops at `/trekplan` until approved. No producer has to emit anything new: the operator approves through `/trekbrief --approve <project-dir | brief-file>`, a Voyage-side step open to every producer's brief, whether it is planned with `--project` (`<dir>/brief.md`) or `--brief <file>`, so no producer is privileged. `trekreview` briefs are exempt. The exemption is read from the brief's own frontmatter, so relabelling a brief `type: trekreview` walks past the gate, the same trust class as self-stamping. **What the marker does not prove:** same user, same machine. Any session can run `--stamp`. The marker is a trace that the approval step ran against this exact text, not a signature (see the header of `lib/validators/intent-approval.mjs`).
**Intent approval gate (veikart steg 1) — additive schema, consumer-side requirement.** The two `intent_approved_*` fields are additive and optional in the schema: `brief-validator.mjs` does not require them, `/trekresearch` does not read them, and no `brief_version` bump is involved. The requirement lives in `/trekplan` Phase 1 (`### Read the brief`), which runs `intent-approval.mjs --check` and **halts** on `BRIEF_INTENT_NOT_APPROVED` (no marker), `BRIEF_INTENT_APPROVAL_STALE` (`## Intent`, `## Goal` or `## Threat Model` changed after approval) or `BRIEF_INTENT_APPROVAL_INVALID` (malformed marker, or a section missing, empty or present more than once; a `## ` line inside a code block or HTML comment is not a heading). It is version-independent, so a brief from ANY producer, of any `brief_version`, stops at `/trekplan` until approved. No producer has to emit anything new: the operator approves through `/trekbrief --approve <project-dir | brief-file>`, a Voyage-side step open to every producer's brief, whether it is planned with `--project` (`<dir>/brief.md`) or `--brief <file>`, so no producer is privileged. `trekreview` briefs are exempt. The exemption is read from the brief's own frontmatter, so relabelling a brief `type: trekreview` walks past the gate, the same trust class as self-stamping. **What the marker does not prove:** same user, same machine. Any session can run `--stamp`. The marker is a trace that the approval step ran against this exact text, not a signature (see the header of `lib/validators/intent-approval.mjs`).
**Approval log (K2) — consumer-side requirement, no schema change.** After the stamp, `/trekbrief` appends the approval to the target repo's log with `lib/util/approval-log.mjs append --kind intent --subject <brief> --sha256 <intent hash>` and commits it with `lib/util/artifact-commit.mjs` (the log always; the brief too when `artifact_store: git`). The commit carries the caller's trailer: every call passes `--trailer "Co-Authored-By: Claude {model} <noreply@anthropic.com>"` with `{model}` filled in, and the CLI refuses (exit 1, nothing committed) a missing trailer, a multi-line one, one that does not read `Co-Authored-By: <name> <<email>>` (an unfilled `{model}` included) and one holding `Claude-Session:`; a bad `--store` stays exit 2 and is checked first. The log (`.claude/voyage/approvals.jsonl`, read and written only by that CLI) is append-only and hash-chained: each record carries `seq`, `ts`, `kind` (`intent | plan | review | merge | release`), `subject`, the content's `sha256`, an optional `expires`, `prev` and its own `hash`. `/trekplan` Phase 1 runs `approval-log.mjs verify --kind intent --subject <brief> --sha256 <intent_approved_hash>` after the marker check and halts on `broken` (a record edited, removed or reordered), `diverged` (the committed log is no longer a prefix of the file), `not-logged` or `mismatch`; exit 2 (could not run) halts too. A brief approved before the log existed is `not-logged`: the remedy is the same `/trekbrief --approve`, open to every producer's brief. Only an explicit "Approve" counts: `approval-log.mjs answer` refuses «høres bra ut», «gjør som du vil» and an empty answer. Trust limits are those of the marker (same user, same machine; a trace, not a signature); what the log adds is that each approval is a commit in `git log`, and that a committed approval cannot be rewritten without `diverged`. Adapted from bashebr/ai-native-sdlc@11fb8d7 (MIT), corrected so the approval is bound to content and cannot be written unseen.
@ -117,6 +119,7 @@ Optional but standard sections: `## Non-Goals`, `## Constraints`, `## Preference
- `BRIEF_INTENT_NOT_APPROVED` / `BRIEF_INTENT_APPROVAL_STALE` / `BRIEF_INTENT_APPROVAL_INVALID` → `/trekplan` halts (intent-approval gate, above); remedy `/trekbrief --approve <project-dir | brief-file>`
- approval log `broken` / `diverged` → `/trekplan` halts; restore `.claude/voyage/approvals.jsonl` from git. `not-logged` / `mismatch` → halts; remedy `/trekbrief --approve <project-dir | brief-file>`
- `BRIEF_INVALID_CEREMONY` / `BRIEF_INVALID_ARTIFACT_STORE` → strict halt; the optional field is present with a value outside its enum
- `BRIEF_THREAT_MODEL_MALFORMED` → strict halt; the optional `## Threat Model` is present with no `- Inside:` / `- Outside:` bullet, another label or form, empty text, or twice
- `BRIEF_NOT_FOUND` → consumer halts with a usage message
- `FM_MISSING` → file has no frontmatter; halt
- `BRIEF_WRONG_TYPE` → file is not a brief; halt
@ -467,6 +470,7 @@ Since 1.2 (every field and section optional; a malformed one present is a strict
- `## Resolved since round <n>` — the previous round's findings this round no longer raises; written by `review-finalize.mjs`.
- `## Answers` — the operator's answer to each counting MAJOR, written by `/trekreview` Phase 8.8 after `## Remediation Steps` and before the trailing JSON block: a pipe table with the header exactly `| Finding | Answer | Reason | Record | Carried from |`; per row a 40-hex finding id (optionally `#n`), `accepted` or `fixed`, a reason (non-empty when `accepted`), the 64-hex hash of the K2 `review` record, and `-` or `review-round-<run12>-<n>.md#<id>`; at most one row per finding. A MAJOR answered `fixed` needs its `### Step R<n>` with `Source finding: <id>` under `## Remediation Steps`. Code `REVIEW_ANSWERS_MALFORMED`.
- `## Declined to Judge` gains three reasons: `policy:skip` and `policy:cap` (a MINOR or SUGGESTION finding the `REVIEW.md` policy skipped or capped) and `instruction-out-of-set` (a CLAUDE.md-key finding that cites a file outside the tracked instruction set).
- `known_outside` (E, decisions 164, 218), trailing JSON block, additive: Voyage's own security findings (no `engine`, a security key) that quote an `Outside:` line of the brief's `## Threat Model` in `threat_model_ref`. The coordinator writes them there, never into `findings`; they do not count. `review-finalize.mjs` checks each against the brief at `brief_path` and puts back into the findings, at its catalogue tier, any the brief does not back now (the verdict only rises); the rest are listed under `## Outside the Threat Model`. The verdict gate (`decideOpenFindings`, so the release check too) re-checks each on every run and exits 2 for one not backed. A malformed list or entry (an entry needs a string `rule_key` and `file`, an integer `line` ≥ 0 and a non-empty `threat_model_ref`) is `REVIEW_KNOWN_OUTSIDE_MALFORMED`.
**The brief-conflict mark (fields `engine`, `contradicts_brief`, `brief_lowered_from`).**
Each finding may carry three more fields, in its body field list and in the trailing JSON block:

View file

@ -2,7 +2,7 @@
// One deterministic pass over review.md after the coordinator and after
// lib/review/fill-finding-ids.mjs, on both paths (review_version 1.2, fix
// round 4.5b2 part B). Its passes run in the Architecture order (engine,
// instruction, carry, pre-existing, policy), each at its place in PASSES:
// outside, instruction, carry, pre-existing, policy), each at its place in PASSES:
//
// - engine, always the first pass: `putBack` (lib/review/engine-compare.mjs)
// compares the engine payloads Phase 4.7 saved in the project folder
@ -17,6 +17,18 @@
// the frontmatter list, counts, a verdict that is only raised, and, as a
// BLOCKER, its `### Step R<n>`. An engine finding in `out_of_scope` is
// not compared (it stays declined, `engine:out-of-scope`).
// - outside, right after engine (E 2.5, decisions 164, 218): the trailing
// JSON block's optional `known_outside` (Voyage's own security findings
// the coordinator listed as outside the brief's threat model) is checked
// against the brief at the review's `brief_path`
// (lib/review/threat-model.mjs `reviewThreatModel`, `backedLine`). It only
// raises: an entry the brief does not back now (its `Outside:` line gone,
// the threat model not usable, an engine finding, not a security finding)
// goes back into the findings at its catalogue tier with no id, a `### `
// entry with a `**Source:**` line saying why, the frontmatter list, counts,
// a verdict that is only raised and, as a BLOCKER, its `### Step R<n>`.
// The entries that stay are listed under `## Outside the Threat Model`;
// a malformed entry stays too (the verdict gate exits 2 on it).
// - instruction, right after engine: a finding under `CLAUDE_MD_DEVIATION`
// or `CLAUDE_MD_STALE` whose `instruction_ref` is missing or names a file
// outside the repo's tracked instruction set (lib/review/instruction-scope.mjs
@ -111,6 +123,8 @@ import { carry, frontmatterField, MAX_REVIEW_ROUNDS, runId } from './review-roun
import { applyPolicy, readPolicy } from './review-policy.mjs';
import { ENGINES } from './engine-findings.mjs';
import { ENGINE_OF_REVIEWER, putBack } from './engine-compare.mjs';
import { backedLine, isKnownEntry, reviewThreatModel } from './threat-model.mjs';
import { getRule } from './rule-catalogue.mjs';
import { isMain } from '../util/is-main.mjs';
const SEVERITIES = ['BLOCKER', 'MAJOR', 'MINOR', 'SUGGESTION'];
@ -243,6 +257,60 @@ function passEngine(payload, ctx) {
return { headEdit: (head) => withVerdict(head, r.raisedVerdict) };
}
// ---- pass: outside -----------------------------------------------------------------
const OUTSIDE_RE = /^##\s+Outside the Threat Model\s*$/;
/** The `## Outside the Threat Model` section: the entries that stay, and how many went back. */
function outsideSection(kept, back) {
const note = back > 0 ? `Put back into the findings, the brief does not back them now: ${back}.\n\n` : '';
if (kept.length === 0) return `## Outside the Threat Model\n\n${note}None.\n\n`;
const items = kept.map((e) => (isKnownEntry(e)
? `- ${oneLine(e.title) || '(no title)'} (\`${oneLine(e.file)}:${e.line}\`, \`${oneLine(e.rule_key)}\`) — Outside: ${oneLine(e.threat_model_ref)}`
: '- (a malformed entry: the verdict gate cannot check it against the brief)'));
return '## Outside the Threat Model\n\n'
+ `Voyage's own security findings the brief's threat model puts outside, listed as known issues; they do not count: ${kept.length}.\n\n`
+ `${note}${items.join('\n')}\n\n`;
}
/**
* Pass `outside`, right after engine: each `known_outside` entry the brief does
* not back now goes back into the findings at its catalogue tier.
*/
function passOutside(payload, ctx) {
if (!Array.isArray(payload.known_outside) || payload.known_outside.length === 0) return {};
const use = reviewThreatModel(ctx.text, ctx.repo);
const tm = use.usable ? use.threatModel : null;
const kept = [];
const back = [];
for (const e of payload.known_outside) {
if (isKnownEntry(e) && !(tm && backedLine(e, tm) !== null)) back.push(e);
else kept.push(e);
}
const why = tm ? 'no `Outside:` line of the brief equals its threat_model_ref, or it is not a Voyage security finding' : use.why;
for (const e of back) {
const rule = getRule(e.rule_key.trim().toUpperCase());
const f = { ...e, id: null, rule_key: rule ? rule.rule_key : e.rule_key, severity: rule ? rule.severity : 'BLOCKER' };
payload.findings.push(f);
ctx.added.add(f);
ctx.outsideBack.set(f, why);
}
payload.known_outside = kept;
ctx.outside = { kept: kept.length, put_back: back.length };
const section = outsideSection(kept, back.length);
const edits = [(head) => replaceSection(head, OUTSIDE_RE, section,
[/^##\s+Pre-existing\s*$/, /^##\s+Resolved since round\b/, /^##\s+Declined to Judge\b/])];
if (back.length > 0) {
ctx.recount = true;
const v = raisedVerdict(payload.verdict, payload.findings);
if (v !== payload.verdict) {
payload.verdict = v;
edits.push((head) => withVerdict(head, v));
}
}
return { headEdit: (head) => edits.reduce((h, fn) => fn(h), head) };
}
// ---- pass: instruction -------------------------------------------------------------
const INSTRUCTION_KEYS = new Set(['CLAUDE_MD_DEVIATION', 'CLAUDE_MD_STALE']);
@ -333,7 +401,7 @@ function passInstruction(payload, ctx) {
// ---- pass: carry -------------------------------------------------------------------
const LATER_SECTIONS = [/^##\s+Suppressed Findings\b/, /^##\s+Pre-existing\s*$/, /^##\s+Resolved since round\b/,
const LATER_SECTIONS = [/^##\s+Suppressed Findings\b/, /^##\s+Outside the Threat Model\s*$/, /^##\s+Pre-existing\s*$/, /^##\s+Resolved since round\b/,
/^##\s+Declined to Judge\b/, /^##\s+Remediation Steps\b/, /^##\s+Remediation Summary\b/, /^##\s+Answers\s*$/];
/** The `## Resolved since round <n>` section. */
@ -531,6 +599,7 @@ function passPolicy(payload, ctx) {
/** The passes, in the Architecture order; the id recomputation runs after them, always last. */
export const PASSES = Object.freeze([
{ name: 'engine', run: passEngine },
{ name: 'outside', run: passOutside },
{ name: 'instruction', run: passInstruction },
{ name: 'carry', run: passCarry },
{ name: 'pre-existing', run: passPreExisting },
@ -589,13 +658,17 @@ function editFrontmatter(head, ids, fields) {
/**
* A body entry for a finding finalize added, in the coordinator's form; an
* engine put-back names its source first.
* engine put-back, or a known issue put back from outside the threat model,
* names its source first.
*/
function entryLines(f, putBackSet) {
function entryLines(f, putBackSet, outsideBack) {
const lines = [`### ${f.id}`, ''];
if (putBackSet.has(f)) lines.push(`**Source:** engine ${oneLine(f.engine)}, put back by the standard-path comparison`, '');
if (outsideBack.has(f)) {
lines.push(`**Source:** listed outside the threat model (Outside: ${oneLine(f.threat_model_ref)}), put back: ${oneLine(outsideBack.get(f))}`, '');
}
lines.push(`- file: ${oneLine(f.file)}`, `- line: ${f.line}`, `- rule_key: ${oneLine(f.rule_key)}`);
for (const k of ['brief_ref', 'title', 'detail', 'recommended_action', 'engine', 'carried_from']) {
for (const k of ['brief_ref', 'title', 'detail', 'recommended_action', 'engine', 'carried_from', 'threat_model_ref']) {
if (f[k] !== undefined && f[k] !== null && oneLine(f[k]) !== '') lines.push(`- ${k}: ${oneLine(f[k])}`);
}
return lines;
@ -612,14 +685,14 @@ function findingsBefore(sev) {
* The body entries: one added for each finding in `added`, and a `### <id>`
* entry that sits under another severity's `## Findings` section moved to its own.
*/
function placeEntries(head, findings, added, putBackSet) {
function placeEntries(head, findings, added, putBackSet, outsideBack = new Map()) {
let lines = head.split('\n');
const fmEnd = frontmatterEnd(lines);
for (const f of findings) {
const sev = sevOf(f);
if (!SEVERITIES.includes(sev) || typeof f.id !== 'string') continue;
if (added.has(f)) {
lines = appendToSection(lines, findingsHeading(sev), `## Findings (${sev})`, entryLines(f, putBackSet), findingsBefore(sev));
lines = appendToSection(lines, findingsHeading(sev), `## Findings (${sev})`, entryLines(f, putBackSet, outsideBack), findingsBefore(sev));
continue;
}
const heading = new RegExp(`^###\\s+${f.id}\\s*$`);
@ -727,7 +800,7 @@ export function finalizeReview(text, { repo, round, roundBase = null, previous =
if (dup !== undefined) return { exit: 1, reason: `two findings carry the id ${JSON.stringify(dup)}` };
const ev = evidence ?? { ran: { engines: [], reviewers: [] }, marks: [], payloads: [] };
const ctx = { repo, text, round, previous, evidence: ev, added: new Set(), putBack: new Set(), carried: false };
const ctx = { repo, text, round, previous, evidence: ev, added: new Set(), putBack: new Set(), outsideBack: new Map(), carried: false };
let head = parts.head;
const out = { passes: [] };
for (const pass of PASSES) {
@ -750,7 +823,7 @@ export function finalizeReview(text, { repo, round, roundBase = null, previous =
head = rewriteIds(head, map);
out.passes.push('ids');
head = placeEntries(head, payload.findings, ctx.added, ctx.putBack);
head = placeEntries(head, payload.findings, ctx.added, ctx.putBack, ctx.outsideBack);
if (ctx.carried || ctx.recount) {
payload.counts = Object.fromEntries(SEVERITIES.map((s) => [s, payload.findings.filter((f) => sevOf(f) === s).length]));
}
@ -778,6 +851,7 @@ export function finalizeReview(text, { repo, round, roundBase = null, previous =
exit: 0, text: `${head}${parts.fence}${body}${parts.tail}`, ...out, ids_rewritten: map.size,
carried: ctx.added.size, resolved: payload.resolved ?? [], ran: payload.ran, r_steps_added: steps.added,
policy: ctx.policy ?? null, put_back: ctx.engine?.put_back ?? 0, instruction: ctx.instruction ?? null,
outside: ctx.outside ?? null,
};
}

View file

@ -21,12 +21,19 @@
// Otherwise every security finding counts, and `why` says why.
// backedLine(finding, tm) -> the `Outside:` text the finding's
// `threat_model_ref` equals exactly, else null.
// reviewThreatModel(reviewText, repo) -> usable() for the brief a review
// names in its `brief_path` (a relative path is read from the repo's top
// level); the review side's `known_outside` (E 2.5) is checked with it.
// isKnownEntry(e) -> a review `known_outside` entry's shape: a
// finding with a string `rule_key` and `file`, an integer `line` >= 0
// and a non-empty `threat_model_ref`.
//
// Lifting a finding back in is the operator's: change (or remove) that
// `Outside:` line and approve the intent again. Both gates re-read the brief on
// every run, so a finding whose line is gone counts in full.
import { readFileSync } from 'node:fs';
import { isAbsolute, join } from 'node:path';
import { parseDocument } from '../util/frontmatter.mjs';
import { verifyLog, normalizeHex, normalizeSubject, resolveRepo } from '../util/approval-log.mjs';
import { checkIntentApprovalContent } from '../validators/intent-approval.mjs';
@ -85,6 +92,31 @@ export function usable(briefPath, repo) {
return { usable: true, why: null, threatModel: tm };
}
/**
* The threat model a review applies: usable() for the brief at the review's
* frontmatter `brief_path`, a relative path read from the repo's top level.
* @param {string} reviewText - review.md
* @param {string} [repo] - the repo (default: the git top level of the working directory)
* @returns {{usable: boolean, why: string|null, threatModel: object}}
*/
export function reviewThreatModel(reviewText, repo) {
const doc = parseDocument(typeof reviewText === 'string' ? reviewText : '');
const bp = doc.parsed?.frontmatter?.brief_path;
if (typeof bp !== 'string' || bp.trim() === '') return { usable: false, why: 'the review names no brief_path', threatModel: EMPTY };
let root;
try { root = resolveRepo(repo); } catch (e) {
return { usable: false, why: `the repo could not be read: ${e.message}`, threatModel: EMPTY };
}
return usable(isAbsolute(bp) ? bp : join(root, bp), root);
}
/** A review `known_outside` entry's shape (E 2.5): string rule_key and file, integer line >= 0, non-empty threat_model_ref. */
export function isKnownEntry(e) {
return e !== null && typeof e === 'object' && !Array.isArray(e) &&
typeof e.rule_key === 'string' && e.rule_key !== '' && typeof e.file === 'string' && e.file !== '' &&
Number.isInteger(e.line) && e.line >= 0 && typeof e.threat_model_ref === 'string' && e.threat_model_ref !== '';
}
/** A security finding that is Voyage's own: a security key, no engine, not the engines' key (218). */
export function isOwnSecurityFinding(f) {
if (!f || typeof f !== 'object') return false;

View file

@ -51,6 +51,14 @@
// An ALLOW/WARN review that holds one finding id twice, in the
// frontmatter `findings:` list or in the trailing JSON block, is exit
// 2 (reason `duplicate finding id <id> in <where>`).
// An ALLOW/WARN review whose optional `known_outside` (E 2.5: Voyage's
// own security findings the brief's threat model puts outside) holds
// an entry the brief at its `brief_path` does not back now (its
// `Outside:` line gone, the threat model not usable, an engine
// finding, not a security finding, a malformed entry or list) is
// exit 2 too (reason `review-not-measured: known issue outside the
// threat model not backed by the brief (<why>): …`;
// `notBackedOutside`): the brief is re-read on every run.
// Exit 2 is NEVER green.
//
// The pure `decideVerdict` stays text-only: it never reads the answers. The
@ -85,6 +93,7 @@ import { frontmatterField, runId } from './review-rounds.mjs';
// No cycle: neither module, nor any module they import, imports this gate.
import { roundEvidence } from './review-finalize.mjs';
import { putBack, normTitle } from './engine-compare.mjs';
import { backedLine, isKnownEntry, reviewThreatModel } from './threat-model.mjs';
export const EXIT = Object.freeze({ PASS: 0, BLOCK: 1, UNDECIDED: 2 });
@ -220,6 +229,7 @@ function blockerSectionEntries(body) {
const NOT_FINALIZED = 'review-not-measured: round 2+ review not finalized';
const MOVED_SERIOUS = 'a policy or filter moved a BLOCKER, MAJOR or security finding';
const DROPPED_ENGINE = 'review-not-measured: engine finding not in review';
const NOT_BACKED = 'review-not-measured: known issue outside the threat model not backed by the brief';
const NOTED_SEVERITIES = new Set(['MINOR', 'SUGGESTION']);
/** One field of a declined entry for the reason line: one line, at most 120 chars. */
@ -307,6 +317,29 @@ export function droppedEngineFindings(text, reviewPath) {
return putBack(review, payloads).added.filter((e) => !carried.some((c) => samePlace(c, e)));
}
/**
* The `known_outside` entries of review.md the brief at its `brief_path` does
* not back now (lib/review/threat-model.mjs `reviewThreatModel`, `backedLine`):
* a malformed list or entry, an engine finding, not a security finding, an
* `Outside:` line gone, or a threat model that is not usable. An unreadable
* trailing block gives none (decideVerdict judges it).
* @param {string} text - review.md
* @param {string|undefined} repo - the repo's top level
* @returns {{entries: Array<*>, why: string|null}}
*/
export function notBackedOutside(text, repo) {
const payload = trailingPayload(String(text ?? ''));
if (payload === null || typeof payload !== 'object' || !('known_outside' in payload)) return { entries: [], why: null };
const list = payload.known_outside;
if (!Array.isArray(list)) return { entries: [list], why: 'known_outside is not a list' };
if (list.length === 0) return { entries: [], why: null };
const use = reviewThreatModel(text, repo);
const tm = use.usable ? use.threatModel : null;
const entries = list.filter((e) => !isKnownEntry(e) || !(tm && backedLine(e, tm) !== null));
const why = tm ? 'no `Outside:` line of the brief equals its threat_model_ref, it is not a Voyage security finding, or it is malformed' : use.why;
return { entries, why: entries.length > 0 ? why : null };
}
/**
* The open-findings check: every counting MAJOR of an ALLOW/WARN review must be
* answered (lib/review/review-answers.mjs `decideAnswers`). K2 records come
@ -322,6 +355,7 @@ export function droppedEngineFindings(text, reviewPath) {
* pre-existing: it keeps its line from the previous round's end. A round 2+
* review that was not finalized (`unfinalizedRound`) is exit 2 first, then an
* engine MAJOR or BLOCKER missing from the review (`droppedEngineFindings`),
* then a `known_outside` entry the brief does not back (`notBackedOutside`),
* then a BLOCKER, MAJOR or security finding moved by a policy or filter (`movedSerious`).
* @param {string} text - review.md
* @param {{ repo?: string, reviewPath: string }} opts
@ -339,6 +373,18 @@ export function decideOpenFindings(text, { repo, reviewPath } = {}) {
fixed_pending: [],
};
}
const outside = notBackedOutside(text, repo);
if (outside.entries.length > 0) {
const what = (e) => (e !== null && typeof e === 'object'
? `${field(e.title) ?? '(no title)'} at ${field(e.file) ?? '?'}:${field(e.line) ?? '?'}`
: JSON.stringify(e));
return {
exit: EXIT.UNDECIDED,
reason: `${NOT_BACKED} (${outside.why}): ${outside.entries.map(what).join(', ')}`,
unanswered: [],
fixed_pending: [],
};
}
const hidden = movedSerious(text);
if (hidden.length > 0) {
return {

View file

@ -22,6 +22,7 @@ import { parseDocument } from '../util/frontmatter.mjs';
import { issue, ok, fail } from '../util/result.mjs';
import { isMain } from '../util/is-main.mjs';
import { extractFindingsBlock } from '../review/findings-schema.mjs';
import { isKnownEntry } from '../review/threat-model.mjs';
export const REVIEW_REQUIRED_FRONTMATTER = [
'type',
@ -76,8 +77,10 @@ export function remediationStepSources(body) {
// `fixed` without its R-step), REVIEW_POLICY_MALFORMED (`review_policy`,
// `review_policy_sha256`, a finding's `instruction_ref`), REVIEW_ROUND_MALFORMED
// (`round`, `round_base`, `finalized`, `resolved`, `ran`, a finding's
// `carried_from`) and REVIEW_PRE_EXISTING_MALFORMED (a finding's `pre_existing`,
// `pre_existing_counts`).
// `carried_from`), REVIEW_PRE_EXISTING_MALFORMED (a finding's `pre_existing`,
// `pre_existing_counts`) and REVIEW_KNOWN_OUTSIDE_MALFORMED (the optional
// `known_outside` list, E 2.5: Voyage's own security findings the brief's
// threat model puts outside; lib/review/threat-model.mjs `isKnownEntry`).
const FULL_SHA_RE = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/;
const SHA256_RE = /^[0-9a-f]{64}$/;
@ -163,6 +166,7 @@ function review12Issues(fm, body) {
const policy = (msg) => out.push(issue('REVIEW_POLICY_MALFORMED', msg));
const pre = (msg) => out.push(issue('REVIEW_PRE_EXISTING_MALFORMED', msg));
const answers = (msg) => out.push(issue('REVIEW_ANSWERS_MALFORMED', msg));
const outside = (msg) => out.push(issue('REVIEW_KNOWN_OUTSIDE_MALFORMED', msg));
// Frontmatter.
if ('round' in fm && !(Number.isInteger(fm.round) && fm.round >= 1 && fm.round <= 3)) {
@ -211,6 +215,17 @@ function review12Issues(fm, body) {
round(`resolved must be an array of 40-char hex finding ids, got ${JSON.stringify(r)}`);
}
}
if ('known_outside' in payload) {
const k = payload.known_outside;
if (!Array.isArray(k)) outside(`known_outside must be an array, got ${JSON.stringify(k)}`);
else {
k.forEach((e, i) => {
if (!isKnownEntry(e)) {
outside(`known_outside[${i}] needs a string rule_key and file, an integer line >= 0 and a non-empty threat_model_ref, got ${JSON.stringify(e)}`);
}
});
}
}
if ('ran' in payload) {
const r = payload.ran;
const okRan = isPlainObject(r)

View file

@ -102,6 +102,12 @@ should look at first.
bug but worth noting for v1.1 cleanup.
- recommended_action: Track in TODO.md if not already.
## Outside the Threat Model
<!-- Optional (E). Written by review-finalize.mjs when the coordinator listed Voyage's own security findings in the JSON block's `known_outside` (each quotes an `Outside:` line of the brief's `## Threat Model` in `threat_model_ref`): the ones the brief still backs, as known issues that do not count; any it does not back now are put back into the findings. -->
None.
## Pre-existing
<!-- Optional (review_version 1.2). Written by review-finalize.mjs: the titles and counts of the findings on lines this range did not add or change (`pre_existing: true`). -->
@ -167,6 +173,7 @@ plan-input.
"ran": { "engines": ["code-review", "security-review"], "reviewers": ["brief-conformance-reviewer", "code-correctness-reviewer"] },
"allow_blocked_by": [],
"declined_to_judge": [],
"known_outside": [],
"trimmed": 0,
"findings": [
{

View file

@ -0,0 +1,341 @@
// tests/lib/review-threat-model.test.mjs
// Part 2, step 2.5 of the E plan (decisions 164, 167, 218): the brief's threat
// model on the review side.
//
// - The coordinator puts a Voyage security finding with a `threat_model_ref`
// into the trailing JSON block's `known_outside`, never into `findings`.
// - lib/review/review-finalize.mjs only raises: an entry the brief does not
// back now (its Outside: line gone, the approval stale, an engine finding,
// not a security finding) goes back into `findings` at its catalogue tier and
// raises the verdict, as the engine put-back does; the entries that stay are
// listed under `## Outside the Threat Model`.
// - lib/review/verdict-gate.mjs `decideOpenFindings` re-checks every entry
// against the review's `brief_path` and exits 2 for one not backed.
// - lib/validators/review-validator.mjs shape-checks the optional field.
// - Prose: the code reviewer's dimension 6, the coordinator, /trekreview Phase
// 8.8, the review template, Handover 1 and 6.
// Tests run the real CLIs (finalize, validator, gate) in throwaway git repos
// with no hooks, a stamped brief and its logged approval.
import { after, test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { appendFileSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { validateReviewContent } from '../../lib/validators/review-validator.mjs';
import { stampIntentApproval } from '../../lib/validators/intent-approval.mjs';
import { appendApproval, normalizeHex, normalizeSubject, resolveRepo } from '../../lib/util/approval-log.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const FINALIZE = join(ROOT, 'lib', 'review', 'review-finalize.mjs');
const VALIDATOR = join(ROOT, 'lib', 'validators', 'review-validator.mjs');
const GATE = join(ROOT, 'lib', 'review', 'verdict-gate.mjs');
const PROJECT_REL = '.claude/projects/2026-11-08-tm';
const read = (rel) => readFileSync(join(ROOT, rel), 'utf8');
const flat = (s) => s.replace(/\s+/g, ' ');
const OUTSIDE = "a local user who can already edit the repo's hooks";
const INSIDE = 'a crafted file name in a pull request reaches the shell';
const BRIEF = `---
type: trekbrief
brief_version: "2.2"
created: 2026-11-08
task: "Add an upload hook"
slug: tm
project_dir: ${PROJECT_REL}/
research_topics: 0
research_status: complete
framing: new-direction
---
# Task
## TL;DR
An upload hook.
## Intent
Uploads arrive from strangers.
## Goal
A hook that checks each upload.
## Threat Model
- Inside: ${INSIDE}
- Outside: ${OUTSIDE}
## Success Criteria
- It checks.
`;
const made = [];
after(() => { for (const b of made) rmSync(b, { recursive: true, force: true }); });
function g(cwd, ...args) {
const r = spawnSync('git', args, { cwd, encoding: 'utf8', env: childEnv() });
if (r.status !== 0) throw new Error(`git ${args.join(' ')}: ${r.stderr}`);
return r.stdout.trim();
}
const run = (args) => spawnSync(process.execPath, args, { encoding: 'utf8', env: childEnv() });
/** A throwaway repo with a start and an end commit, a stamped brief and its logged intent approval. */
function tmRepo() {
const base = mkdtempSync(join(tmpdir(), 'review-tm-'));
made.push(base);
const repo = join(base, 'repo');
mkdirSync(repo, { recursive: true });
mkdirSync(join(base, 'hooks-none'));
g(repo, 'init', '-q', '-b', 'main');
appendFileSync(join(repo, '.git', 'config'), '[user]\n\tname = Test\n\temail = test@example.invalid\n'
+ `[commit]\n\tgpgsign = false\n[core]\n\thooksPath = "${join(base, 'hooks-none')}"\n`);
appendFileSync(join(repo, '.git', 'info', 'exclude'), '.claude/\n');
mkdirSync(join(repo, 'commands'), { recursive: true });
writeFileSync(join(repo, 'commands', 'up.md'), '# up\n\none\ntwo\n');
g(repo, 'add', '-A');
g(repo, 'commit', '-q', '-m', 'chore: start');
const start = g(repo, 'rev-parse', 'HEAD');
writeFileSync(join(repo, 'commands', 'up.md'), '# up\n\none\n```bash\nrm -rf "$NAME"\n```\n');
g(repo, 'add', '-A');
g(repo, 'commit', '-q', '-m', 'feat: the work');
const end = g(repo, 'rev-parse', 'HEAD');
const project = join(repo, ...PROJECT_REL.split('/'));
mkdirSync(project, { recursive: true });
const briefPath = join(project, 'brief.md');
const s = stampIntentApproval(BRIEF, new Date('2026-11-08T08:00:00Z'));
assert.equal(s.stamped, true);
writeFileSync(briefPath, s.text);
const root = resolveRepo(repo);
assert.equal(appendApproval({ repo: root, kind: 'intent', subject: normalizeSubject(root, briefPath), sha: normalizeHex(s.hash) }).appended, true);
const reviewPath = join(project, 'review.md');
return {
repo, start, end, project, briefPath, reviewPath,
write: (text) => writeFileSync(reviewPath, text),
read: () => readFileSync(reviewPath, 'utf8'),
finalize: () => run([FINALIZE, '--review', reviewPath, '--repo', repo, '--project', project, '--round', '1']),
validate: () => run([VALIDATOR, reviewPath]),
gate: () => run([GATE, '--json', reviewPath, '--repo', repo]),
};
}
/** An entry as the coordinator writes it into `known_outside`. */
const entry = (extra = {}) => ({
severity: 'BLOCKER', rule_key: 'SECURITY_RISK', file: 'commands/up.md', line: 5,
brief_ref: 'Threat Model', title: 'A hook edited by a local user runs the removal',
detail: 'commands/up.md:5 runs rm on a name a local user can set in a hook.',
recommended_action: 'Quote the name and refuse a path outside the project.',
threat_model_ref: OUTSIDE, ...extra,
});
/** A clean ALLOW review_version 1.2 review of start..end with `known` in known_outside. */
function tmReview({ start, end, known }) {
const json = JSON.stringify({
verdict: 'ALLOW', counts: { BLOCKER: 0, MAJOR: 0, MINOR: 0, SUGGESTION: 0 },
allow_blocked_by: [], declined_to_judge: [], trimmed: 0, findings: [],
...(known !== undefined ? { known_outside: known } : {}),
}, null, 2);
return `---
type: trekreview
review_version: "1.2"
task: "t"
slug: tm
project_dir: ${PROJECT_REL}/
brief_path: ${PROJECT_REL}/brief.md
scope_sha_start: ${start}
scope_sha_end: ${end}
reviewed_files_count: 1
verdict: ALLOW
findings: []
---
# Review: t
## Executive Summary
x
## Coverage
| File | Treatment | Reason |
|------|-----------|--------|
| commands/up.md | deep-review | default |
## Declined to Judge
None.
Trimmed: 0
## Remediation Summary
x
\`\`\`json
${json}
\`\`\`
`;
}
const jsonOf = (text) => JSON.parse(text.slice(text.lastIndexOf('```json\n') + 8, text.lastIndexOf('```')));
const fmVerdict = (text) => /^verdict:\s*(\S+)/m.exec(text)?.[1];
function sectionOf(text, heading) {
const start = text.indexOf(`\n${heading}`);
if (start === -1) return '';
const next = text.indexOf('\n## ', start + heading.length + 1);
return text.slice(start + 1, next === -1 ? undefined : next);
}
const codes = (text) => validateReviewContent(text, { strict: true }).errors.map((e) => e.code);
// ---- the validator ------------------------------------------------------------------
test('validator: a well-formed known_outside is valid; a malformed one is REVIEW_KNOWN_OUTSIDE_MALFORMED (strict)', () => {
const ok = tmReview({ start: 'a'.repeat(40), end: 'b'.repeat(40), known: [entry()] });
assert.deepEqual(codes(ok).filter((c) => c === 'REVIEW_KNOWN_OUTSIDE_MALFORMED'), []);
for (const bad of ['x', [null], [entry({ threat_model_ref: '' })], [entry({ threat_model_ref: undefined })],
[entry({ rule_key: 7 })], [entry({ file: undefined })], [entry({ line: -1 })]]) {
const text = tmReview({ start: 'a'.repeat(40), end: 'b'.repeat(40), known: bad });
assert.ok(codes(text).includes('REVIEW_KNOWN_OUTSIDE_MALFORMED'), `malformed: ${JSON.stringify(bad)}`);
}
});
// ---- finalize -----------------------------------------------------------------------
test('finalize: a backed entry stays in known_outside, counts nothing, and is listed under ## Outside the Threat Model', () => {
const fx = tmRepo();
fx.write(tmReview({ start: fx.start, end: fx.end, known: [entry()] }));
const r = fx.finalize();
assert.equal(r.status, 0, r.stdout + r.stderr);
const text = fx.read();
const json = jsonOf(text);
assert.deepEqual(json.findings, []);
assert.equal(json.known_outside?.length, 1);
assert.equal(json.verdict, 'ALLOW');
const sec = sectionOf(text, '## Outside the Threat Model');
assert.ok(sec.includes(entry().title), sec);
assert.ok(sec.includes(OUTSIDE), sec);
const gate = fx.gate();
assert.equal(gate.status, 0, gate.stdout + gate.stderr);
});
test('finalize: an entry the brief no longer backs goes back into findings at its catalogue tier and raises the verdict', () => {
const fx = tmRepo();
fx.write(tmReview({ start: fx.start, end: fx.end, known: [entry()] }));
// The operator lifts it back in: the Outside line changes (the approval goes stale too).
writeFileSync(fx.briefPath, readFileSync(fx.briefPath, 'utf8').replace(OUTSIDE, 'anyone at all'));
const r = fx.finalize();
assert.equal(r.status, 0, r.stdout + r.stderr);
const text = fx.read();
const json = jsonOf(text);
const back = json.findings.filter((f) => f.rule_key === 'SECURITY_RISK' && f.file === 'commands/up.md');
assert.equal(back.length, 1, `put back: ${JSON.stringify(json.findings)}`);
assert.equal(back[0].severity, 'BLOCKER', 'at its catalogue tier');
assert.match(back[0].id, /^[0-9a-f]{40}$/);
assert.deepEqual(json.known_outside ?? [], []);
assert.equal(json.verdict, 'BLOCK');
assert.equal(fmVerdict(text), 'BLOCK');
assert.equal(json.counts.BLOCKER, 1);
const blockers = sectionOf(text, '## Findings (BLOCKER)');
assert.ok(blockers.includes(`### ${back[0].id}`), blockers);
assert.match(blockers, /\*\*Source:\*\* .*outside the threat model.*put back/i);
assert.ok(sectionOf(text, '## Remediation Steps').includes(back[0].id), 'the put-back BLOCKER gets its R-step');
const v = fx.validate();
assert.equal(v.status, 0, v.stdout + v.stderr);
assert.equal(fx.gate().status, 1, 'BLOCK');
});
test('finalize: an engine finding or a finding that is not a security finding is never backed, whatever its ref', () => {
const fx = tmRepo();
fx.write(tmReview({ start: fx.start, end: fx.end, known: [
entry({ engine: 'security-review', rule_key: 'SECURITY_INJECTION', title: 'engine one' }),
entry({ rule_key: 'MISSING_TEST', severity: 'MAJOR', line: 4, title: 'not security' }),
] }));
const r = fx.finalize();
assert.equal(r.status, 0, r.stdout + r.stderr);
const json = jsonOf(fx.read());
assert.deepEqual(json.findings.map((f) => [f.title, f.severity]).sort(), [['engine one', 'BLOCKER'], ['not security', 'MAJOR']]);
assert.deepEqual(json.known_outside ?? [], []);
assert.equal(json.verdict, 'BLOCK');
});
// ---- the verdict gate ---------------------------------------------------------------
test('gate: an ALLOW review whose known_outside entry the brief does not back is exit 2, never green', () => {
const fx = tmRepo();
fx.write(tmReview({ start: fx.start, end: fx.end, known: [entry({ threat_model_ref: INSIDE })] }));
const g2 = fx.gate();
assert.equal(g2.status, 2, g2.stdout + g2.stderr);
assert.match(JSON.parse(g2.stdout).reason, /threat model/i);
// Control: the same review with a backed entry passes.
fx.write(tmReview({ start: fx.start, end: fx.end, known: [entry()] }));
const g0 = fx.gate();
assert.equal(g0.status, 0, g0.stdout + g0.stderr);
});
test('gate: a backed entry turns not backed once the brief\'s approval is stale (the brief is re-read every time)', () => {
const fx = tmRepo();
fx.write(tmReview({ start: fx.start, end: fx.end, known: [entry()] }));
assert.equal(fx.gate().status, 0);
writeFileSync(fx.briefPath, readFileSync(fx.briefPath, 'utf8').replace(INSIDE, 'a crafted name'));
const g2 = fx.gate();
assert.equal(g2.status, 2, g2.stdout + g2.stderr);
assert.match(JSON.parse(g2.stdout).reason, /BRIEF_INTENT_APPROVAL_STALE/);
});
// ---- prose --------------------------------------------------------------------------
function subsection(text, heading) {
const start = text.indexOf(heading);
assert.ok(start >= 0, `heading missing: ${heading}`);
const rest = text.slice(start + heading.length);
const next = rest.search(/\n#{2,3} /);
return text.slice(start, next === -1 ? undefined : start + heading.length + next);
}
test('code-correctness-reviewer: dimension 6 reads only what the engines do not (markdown carrying shell), with the threshold, the keys and threat_model_ref', () => {
const text = read('agents/code-correctness-reviewer.md');
const dim = flat(subsection(text, '### 6. '));
assert.match(dim, /command, skill and agent markdown/);
assert.match(dim, /concrete attack path/);
assert.match(dim, /high confidence[^.]*Claude Code's (own )?threshold/);
assert.match(dim, /`SECURITY_INJECTION`/);
assert.match(dim, /`SECURITY_RISK`/);
assert.match(dim, /`threat_model_ref`/);
assert.match(dim, /`Outside:`/);
assert.doesNotMatch(dim, /XSS|SQL query/, 'code-level security is the engines\' part now');
assert.ok(!text.includes('SECURITY_REVIEW_VULNERABILITY'), 'the reviewer never names the engines\' key');
assert.ok(text.split('\n').some((l) => l.startsWith('| `SECURITY_RISK` | BLOCKER |')), 'the severity table names SECURITY_RISK');
});
test('review-coordinator: a Voyage security finding with threat_model_ref goes to known_outside, never into findings', () => {
const t = flat(read('agents/review-coordinator.md'));
assert.match(t, /`threat_model_ref`[^.]*`known_outside`[^.]*never into `findings`/);
assert.match(t, /"known_outside"/, 'the JSON template carries known_outside');
});
test('/trekreview Phase 8.8 lists the known issues outside the threat model; the template shows the section and field', () => {
const md = read('commands/trekreview.md');
const p88 = flat(md.slice(md.indexOf('## Phase 8.8'), md.indexOf('## Phase 9')));
assert.match(p88, /## Outside the Threat Model/);
assert.match(p88, /`known_outside`/);
assert.match(p88, /not answered/);
const tpl = read('templates/trekreview-template.md');
assert.match(tpl, /^## Outside the Threat Model$/m);
assert.match(tpl, /"known_outside": \[/);
});
test('Handover 1 names the optional ## Threat Model; Handover 6 names known_outside and its code', () => {
const h = read('docs/HANDOVER-CONTRACTS.md');
const h1 = h.slice(h.indexOf('## Handover 1'), h.indexOf('## Handover 2'));
assert.match(h1, /`## Threat Model`/);
assert.match(h1, /BRIEF_THREAT_MODEL_MALFORMED/);
const h6 = h.slice(h.indexOf('## Handover 6'), h.indexOf('## Handover 7'));
assert.match(h6, /`known_outside`/);
assert.match(h6, /REVIEW_KNOWN_OUTSIDE_MALFORMED/);
assert.match(h6, /## Outside the Threat Model/);
});