feat(brief): optional threat model in the brief (decision 164, 218)
A brief may carry an optional `## Threat Model` with one line per bullet, `- Inside: <text>` or `- Outside: <text>`. Present and malformed (no such bullet, another label or form, empty text, two sections) is BRIEF_THREAT_MODEL_MALFORMED; absent is today's rule. No brief_version bump (the ceremony / artifact_store precedent). Sections are found fence- and comment-aware through intent-approval's new sectionTexts, so the validator and the hash count the same sections; brief-validator exports parseThreatModel for the reader. The intent hash covers `## Threat Model` when present (two sections -> null hash, as Intent and Goal), so an edited, added or removed threat model makes the approval stale; a brief without one hashes exactly as before. The stale message and the /trekplan halt table name the section. /trekbrief offers the section in its question round when the task touches a guard, a gate, untrusted input, secrets or publishing, writes it only when given, shows it verbatim at the Phase 4h approval, and --approve reads it out. The template documents it; brief-gate-coverage pairs the new code. Part 2, step 2.1 of the E plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
07a3c06c57
commit
74f75dcb7d
SSH key fingerprint:
SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
7 changed files with 350 additions and 23 deletions
|
|
@ -89,7 +89,7 @@ Usage: /trekbrief <task description>
|
|||
Modes:
|
||||
default Dynamic interview until quality gates pass — brief with research plan
|
||||
--quick Compact start; still escalates on weak sections — brief with research plan
|
||||
--approve Approve an existing brief's ## Intent + ## Goal so /trekplan will plan it
|
||||
--approve Approve an existing brief's ## Intent + ## Goal (+ ## Threat Model) so /trekplan will plan it
|
||||
|
||||
Examples:
|
||||
/trekbrief Add user authentication with JWT tokens
|
||||
|
|
@ -210,7 +210,8 @@ The round runs in the form of `voyage:interviewing`. What `/trekbrief` adds:
|
|||
Success Criteria, Research Plan, Non-Goals, Constraints, Preferences, NFRs,
|
||||
Prior Attempts), plus three fixed questions every round carries: framing
|
||||
(Phase 2.5, first), the size class (Step 3b) and where the brief is stored
|
||||
(`artifact_store`, below).
|
||||
(`artifact_store`, below); and the threat model (below) when the task
|
||||
touches a guard, a gate, untrusted input, secrets or publishing.
|
||||
2. **The budget is the class budget** (table below). A `bounded` or
|
||||
`architectural` round is consecutive `AskUserQuestion` calls with no
|
||||
drafting in between.
|
||||
|
|
@ -247,6 +248,17 @@ the repo is public or you cannot tell:
|
|||
The approval log is committed either way; it holds hashes, times and file
|
||||
names, never the text.
|
||||
|
||||
**The threat model** (`## Threat Model`, optional; E, decision 164). Offer it
|
||||
as one of the round's questions only when the task touches a guard, a gate,
|
||||
untrusted input, secrets or publishing; otherwise the brief has none and every
|
||||
security finding counts. Recommend what is inside (what the work must defend
|
||||
against) and what is outside (what it deliberately does not), one line each.
|
||||
Voyage's own security findings (plan-critic, scope-guardian, the code
|
||||
reviewer's part for command texts) block inside it; one that quotes an
|
||||
`Outside:` line becomes a listed known issue instead of a blocker. Findings
|
||||
from the security engine block as before. The intent approval (Phase 4h)
|
||||
covers the section.
|
||||
|
||||
**Borrowed, with credit.** The size classes, "heaviest in doubt" and the
|
||||
one-way ratchet: obra/superpowers@8ca22db (MIT, © 2025 Jesse Vincent),
|
||||
skills/brainstorming. The class ladder: Kiro's docs (kiro.dev — Plan mode,
|
||||
|
|
@ -272,6 +284,7 @@ state = {
|
|||
prior_attempts: { content: "", probes: 0 }, # optional
|
||||
ceremony: "small | bounded | architectural", # Step 3b; only moves up
|
||||
artifact_store: "git | local", # Step 3c
|
||||
threat_model: { inside: [], outside: [] }, # Step 3c; optional, only when offered
|
||||
assumptions: [], # predicted, hollow or force-stopped answers
|
||||
deferred: [], # over budget, with why
|
||||
question_history: [] # list of questions asked
|
||||
|
|
@ -503,6 +516,11 @@ Build the brief text from Phase 3 state by filling the template:
|
|||
(what you said apart from what the operator said); and the Deferred list,
|
||||
one line of why each.
|
||||
- **Prior Attempts:** from state, or "None — fresh task."
|
||||
- **Threat Model:** only when the operator gave one in the round (Step 3c):
|
||||
`## Threat Model` with one line per bullet, each `- Inside: <text>` or
|
||||
`- Outside: <text>`, and nothing else as a bullet (the validator blocks any
|
||||
other bullet with `BRIEF_THREAT_MODEL_MALFORMED`). Otherwise leave the
|
||||
section out. Optional in the schema, so no `brief_version` bump.
|
||||
|
||||
**Step 4b — Write draft to disk**
|
||||
|
||||
|
|
@ -715,8 +733,9 @@ criterion runs plain `bash`: offer `/bin/bash`).
|
|||
brief is validated, run `scripts/annotate.mjs` to build the page where the
|
||||
operator writes their own notes. The loop that follows is the skill
|
||||
`voyage:annotating`: call it when the operator pastes the notes back (a
|
||||
Question is answered, not applied; after Phase 4h, a note on `## Intent` or
|
||||
`## Goal` makes the approval stale, and that is said before the edit).
|
||||
Question is answered, not applied; after Phase 4h, a note on `## Intent`,
|
||||
`## Goal` or `## Threat Model` makes the approval stale, and that is said
|
||||
before the edit).
|
||||
|
||||
```bash
|
||||
ANNOT_HTML=$(node ${CLAUDE_PLUGIN_ROOT}/scripts/annotate.mjs "{PROJECT_DIR}/brief.md" 2>&1)
|
||||
|
|
@ -759,16 +778,18 @@ is given. It runs on EVERY brief, before the Phase 5 manual/auto fork, so
|
|||
both paths pass through it — and it is **asked even in `--quick` mode**; there
|
||||
is no skip path and no default answer.
|
||||
|
||||
1. Show the operator the `## Intent` and `## Goal` sections of
|
||||
`{BRIEF_PATH}` **verbatim** — the exact text the approval will
|
||||
bind to. Do not summarize them.
|
||||
1. Show the operator the `## Intent`, `## Goal` and, when present,
|
||||
`## Threat Model` sections of `{BRIEF_PATH}` **verbatim** — the exact text
|
||||
the approval will bind to. Do not summarize them. In `--approve` mode too:
|
||||
a brief from another producer may carry a `## Threat Model`, and it is read
|
||||
out with the rest, since the approval covers it.
|
||||
2. Ask via `AskUserQuestion` — one question:
|
||||
**"Is this the intent /trekplan should plan against?"**
|
||||
|
||||
| Option | Effect |
|
||||
|--------|--------|
|
||||
| **Approve** | Check the answer (step 3), then stamp, log and commit (steps 4–6). |
|
||||
| **Revise first** | The operator states the change; edit `## Intent` / `## Goal` in `{BRIEF_PATH}`, re-run the Phase 4g validator, then return to step 1. |
|
||||
| **Revise first** | The operator states the change; edit `## Intent` / `## Goal` / `## Threat Model` in `{BRIEF_PATH}`, re-run the Phase 4g validator, then return to step 1. |
|
||||
| **Leave unapproved** | Write nothing. `/trekplan` will halt on this brief until `/trekbrief --approve {BRIEF_PATH}` is run. |
|
||||
|
||||
3. **Is the answer an approval?** Only an explicit "Approve" is. Check the
|
||||
|
|
@ -794,9 +815,9 @@ node ${CLAUDE_PLUGIN_ROOT}/lib/validators/intent-approval.mjs --stamp "{BRIEF_PA
|
|||
```
|
||||
|
||||
This writes `intent_approved_hash` (sha256 over the normalized `## Intent`
|
||||
+ `## Goal`) and `intent_approved_at` into the frontmatter. It prints JSON; if `stamped` is `false`
|
||||
+ `## Goal`, + `## Threat Model` when present) and `intent_approved_at` into the frontmatter. It prints JSON; if `stamped` is `false`
|
||||
(e.g. `## Goal` is empty), report the `reason` and return to step 1. Editing
|
||||
`## Intent` or `## Goal` afterwards makes the approval stale — `/trekplan`
|
||||
`## Intent`, `## Goal` or `## Threat Model` afterwards makes the approval stale — `/trekplan`
|
||||
halts until the operator approves the new text.
|
||||
|
||||
5. **Log the approval** (K2), whatever `artifact_store` says. `{INTENT_HASH}`
|
||||
|
|
|
|||
|
|
@ -197,9 +197,9 @@ agent, write no plan. Print each error's `message` and `hint`, then stop:
|
|||
|
||||
| Code | Meaning | What the operator does |
|
||||
|------|---------|------------------------|
|
||||
| `BRIEF_INTENT_NOT_APPROVED` | No approval marker (`intent_approved_hash`) in the brief | `/trekbrief --approve {brief_path}` — read `## Intent` + `## Goal`, answer "Approve" |
|
||||
| `BRIEF_INTENT_APPROVAL_STALE` | `## Intent` or `## Goal` changed after approval | Same command — the approval covers the old text, not this one |
|
||||
| `BRIEF_INTENT_APPROVAL_INVALID` | Marker malformed, or `## Intent` / `## Goal` missing, empty or present more than once | Fix the brief, then the same command |
|
||||
| `BRIEF_INTENT_NOT_APPROVED` | No approval marker (`intent_approved_hash`) in the brief | `/trekbrief --approve {brief_path}` — read `## Intent` + `## Goal` (+ `## Threat Model` when present), answer "Approve" |
|
||||
| `BRIEF_INTENT_APPROVAL_STALE` | `## Intent`, `## Goal` or `## Threat Model` changed (or a `## Threat Model` was added or removed) after approval | Same command — the approval covers the old text, not this one |
|
||||
| `BRIEF_INTENT_APPROVAL_INVALID` | Marker malformed, or `## Intent` / `## Goal` missing, empty or present more than once, or `## Threat Model` present more than once | Fix the brief, then the same command |
|
||||
|
||||
If the command does not print the JSON report (module not found, node
|
||||
missing, an unresolved `${CLAUDE_PLUGIN_ROOT}`), **halt** as well and show the
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ import { parseDocument } from '../util/frontmatter.mjs';
|
|||
import { issue, ok, fail } from '../util/result.mjs';
|
||||
import { parseSuccessCriteria, runnableCheck, firstCommand } from '../verification/criteria-runner.mjs';
|
||||
import { isMain } from '../util/is-main.mjs';
|
||||
import { sectionTexts, THREAT_MODEL_SECTION } from './intent-approval.mjs';
|
||||
|
||||
export const BRIEF_REQUIRED_FRONTMATTER = ['type', 'brief_version', 'task', 'slug', 'research_topics', 'research_status'];
|
||||
export const REVIEW_AS_BRIEF_REQUIRED_FRONTMATTER = ['type', 'task', 'slug', 'project_dir', 'findings'];
|
||||
|
|
@ -33,6 +34,43 @@ export const BRIEF_CEREMONY_DEFAULT = 'architectural';
|
|||
export const BRIEF_ARTIFACT_STORE_VALUES = Object.freeze(['git', 'local']);
|
||||
export const BRIEF_ARTIFACT_STORE_DEFAULT = 'local';
|
||||
|
||||
// E 2.1 — the optional `## Threat Model` (decision 164): one line per bullet,
|
||||
// `- Inside: <text>` or `- Outside: <text>`. Additive, no brief_version bump
|
||||
// (the ceremony / artifact_store precedent). Absent = today's rule: every
|
||||
// security finding counts. Sections are found as the intent hash finds them
|
||||
// (fence- and comment-aware, intent-approval.mjs), so the validator, the hash
|
||||
// and the reader count the same sections.
|
||||
const THREAT_BULLET_RE = /^\s*[-*+]\s+/;
|
||||
const THREAT_LINE_RE = /^\s*-\s+(Inside|Outside):(.*)$/;
|
||||
|
||||
/**
|
||||
* Read the optional `## Threat Model` of a brief body.
|
||||
* Every bullet must be `- Inside: <text>` or `- Outside: <text>` with text;
|
||||
* a line that is not a bullet is prose and ignored. Malformed = no such bullet,
|
||||
* a bullet with another label or form, a label with empty text, or two sections.
|
||||
* @param {string} body
|
||||
* @returns {{present: boolean, inside: string[], outside: string[], problems: string[]}}
|
||||
* inside / outside: each bullet's text after the label, trimmed
|
||||
*/
|
||||
export function parseThreatModel(body) {
|
||||
const found = sectionTexts(body, THREAT_MODEL_SECTION);
|
||||
const out = { present: found.length > 0, inside: [], outside: [], problems: [] };
|
||||
if (found.length === 0) return out;
|
||||
if (found.length > 1) out.problems.push(`## ${THREAT_MODEL_SECTION} appears ${found.length} times`);
|
||||
for (const line of found[0].split(/\r?\n/)) {
|
||||
if (!THREAT_BULLET_RE.test(line)) continue;
|
||||
const m = THREAT_LINE_RE.exec(line);
|
||||
if (!m) { out.problems.push(`"${line.trim()}" is not "- Inside: <text>" or "- Outside: <text>"`); continue; }
|
||||
const text = m[2].trim();
|
||||
if (text === '') { out.problems.push(`"${line.trim()}" has no text`); continue; }
|
||||
(m[1] === 'Inside' ? out.inside : out.outside).push(text);
|
||||
}
|
||||
if (out.inside.length === 0 && out.outside.length === 0 && out.problems.length === 0) {
|
||||
out.problems.push('it has no "- Inside: <text>" or "- Outside: <text>" bullet');
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** The brief's size class; absent or invalid → architectural (never lighter in doubt). */
|
||||
export function briefCeremony(fm) {
|
||||
return BRIEF_CEREMONY_VALUES.includes(fm?.ceremony) ? fm.ceremony : BRIEF_CEREMONY_DEFAULT;
|
||||
|
|
@ -99,6 +137,16 @@ export function validateBriefContent(text, opts = {}) {
|
|||
));
|
||||
}
|
||||
|
||||
const threat = parseThreatModel(body);
|
||||
if (threat.problems.length > 0) {
|
||||
errors.push(issue(
|
||||
'BRIEF_THREAT_MODEL_MALFORMED',
|
||||
`## ${THREAT_MODEL_SECTION} is malformed: ${threat.problems.join('; ')}`,
|
||||
'Keep one ## Threat Model with one line per bullet, each "- Inside: <text>" or "- Outside: <text>"; ' +
|
||||
'or leave the section out, and every security finding counts.',
|
||||
));
|
||||
}
|
||||
|
||||
if (typeof fm.brief_version === 'string' || typeof fm.brief_version === 'number') {
|
||||
const vm = String(fm.brief_version).match(/^(\d+)\.(\d+)$/);
|
||||
if (vm) {
|
||||
|
|
|
|||
|
|
@ -5,16 +5,20 @@
|
|||
// approved its intent in /trekbrief (Phase 4h, or `/trekbrief --approve`):
|
||||
//
|
||||
// intent_approved_hash: "sha256:<64 hex>" hash over ## Intent + ## Goal
|
||||
// (+ ## Threat Model when present)
|
||||
// intent_approved_at: "<ISO-8601>" when the stamp was written
|
||||
//
|
||||
// The hash binds the approval to the EXACT intent text: each section is
|
||||
// normalized (lines trimmed, whitespace runs collapsed, blank lines dropped)
|
||||
// and the pair is hashed. Indentation, spacing inside a line, blank lines and
|
||||
// CRLF keep the approval. Line breaks are part of the text:
|
||||
// and the pair is hashed. The optional ## Threat Model (E, decision 164) is
|
||||
// part of what the operator approves: when the brief has one it joins the
|
||||
// hash, so editing, adding or removing it makes the approval stale; a brief
|
||||
// without one hashes exactly as before. Indentation, spacing inside a line,
|
||||
// blank lines and CRLF keep the approval. Line breaks are part of the text:
|
||||
// joining or splitting lines makes the approval stale, as does changing a
|
||||
// word. That errs safe: /trekplan stops until the operator approves the new text.
|
||||
// Each section must appear exactly once; a `## ` line inside a fenced code
|
||||
// block or an HTML comment is text, not a heading.
|
||||
// Each section must appear exactly once (## Threat Model at most once); a `## `
|
||||
// line inside a fenced code block or an HTML comment is text, not a heading.
|
||||
//
|
||||
// WHAT THE MARKER DOES NOT PROVE. Same user, same machine: any session with
|
||||
// write access to the brief can run `--stamp` itself, or write the two fields
|
||||
|
|
@ -48,9 +52,11 @@ import { isMain } from '../util/is-main.mjs';
|
|||
export const INTENT_HASH_FIELD = 'intent_approved_hash';
|
||||
export const INTENT_APPROVED_AT_FIELD = 'intent_approved_at';
|
||||
export const INTENT_SECTIONS = Object.freeze(['Intent', 'Goal']);
|
||||
/** The optional section the intent approval also covers when present (E 2.1). */
|
||||
export const THREAT_MODEL_SECTION = 'Threat Model';
|
||||
const HASH_RE = /^sha256:[0-9a-f]{64}$/;
|
||||
|
||||
const APPROVE_HINT = 'Run /trekbrief --approve <project-dir | brief-file> — the operator reads ## Intent + ## Goal and answers "Approve".';
|
||||
const APPROVE_HINT = 'Run /trekbrief --approve <project-dir | brief-file> — the operator reads ## Intent + ## Goal (+ ## Threat Model) and answers "Approve".';
|
||||
|
||||
/**
|
||||
* Line indexes of the `## ` headings that are real headings: a `## ` line
|
||||
|
|
@ -91,6 +97,20 @@ function extractSections(lines, heads, heading) {
|
|||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Every real `## <heading>` section's text in a brief body, one entry per
|
||||
* occurrence; a `## ` line inside a fenced code block or an HTML comment is
|
||||
* text, not a heading. The brief validator and the threat-model reader use it,
|
||||
* so they count sections exactly as the intent hash does.
|
||||
* @param {string} body
|
||||
* @param {string} heading
|
||||
* @returns {string[]}
|
||||
*/
|
||||
export function sectionTexts(body, heading) {
|
||||
const lines = (body || '').split('\n');
|
||||
return extractSections(lines, headingLines(lines), heading);
|
||||
}
|
||||
|
||||
function normalize(text) {
|
||||
return text.split(/\r?\n/)
|
||||
.map((l) => l.trim().replace(/\s+/g, ' '))
|
||||
|
|
@ -99,9 +119,10 @@ function normalize(text) {
|
|||
}
|
||||
|
||||
/**
|
||||
* Hash ## Intent + ## Goal of a brief body.
|
||||
* @returns {{ hash: string|null, missing: string[], duplicate: string[] }} hash is null when a
|
||||
* section is absent or empty, or appears more than once (which one would the approval cover?).
|
||||
* Hash ## Intent + ## Goal (+ ## Threat Model when present) of a brief body.
|
||||
* @returns {{ hash: string|null, missing: string[], duplicate: string[] }} hash is null when
|
||||
* ## Intent or ## Goal is absent or empty, or any of the three appears more than once (which
|
||||
* one would the approval cover?).
|
||||
*/
|
||||
export function computeIntentHash(body) {
|
||||
const lines = (body || '').split('\n');
|
||||
|
|
@ -116,7 +137,11 @@ export function computeIntentHash(body) {
|
|||
if (!n && found.length <= 1) missing.push(h);
|
||||
parts.push(n);
|
||||
}
|
||||
const threat = extractSections(lines, heads, THREAT_MODEL_SECTION);
|
||||
if (threat.length > 1) duplicate.push(THREAT_MODEL_SECTION);
|
||||
if (missing.length || duplicate.length) return { hash: null, missing, duplicate };
|
||||
// Absent: two parts, the hash every brief had before E. Present: a third part.
|
||||
if (threat.length === 1) parts.push(normalize(threat[0]));
|
||||
const hex = createHash('sha256').update(JSON.stringify(parts), 'utf8').digest('hex');
|
||||
return { hash: `sha256:${hex}`, missing, duplicate };
|
||||
}
|
||||
|
|
@ -163,14 +188,14 @@ export function checkIntentApprovalContent(text) {
|
|||
errors.push(issue(
|
||||
'BRIEF_INTENT_APPROVAL_INVALID',
|
||||
`The approval cannot bind: ${unbindable(bound)}.`,
|
||||
'Leave exactly one non-empty ## Intent and one ## Goal, then ' + APPROVE_HINT,
|
||||
'Leave exactly one non-empty ## Intent and one ## Goal, and at most one ## Threat Model, then ' + APPROVE_HINT,
|
||||
));
|
||||
return result();
|
||||
}
|
||||
if (hash !== recorded) {
|
||||
errors.push(issue(
|
||||
'BRIEF_INTENT_APPROVAL_STALE',
|
||||
'## Intent or ## Goal changed after the intent was approved — the approval covers the old text, not this one.',
|
||||
'## Intent, ## Goal or ## Threat Model changed (or a ## Threat Model was added or removed) after the intent was approved — the approval covers the old text, not this one.',
|
||||
APPROVE_HINT,
|
||||
));
|
||||
}
|
||||
|
|
|
|||
|
|
@ -88,6 +88,20 @@ Quantified where possible.*
|
|||
- {NFR 1 — e.g., "p95 response time < 200ms"}
|
||||
- {NFR 2 — e.g., "Zero new npm dependencies"}
|
||||
|
||||
## Threat Model
|
||||
|
||||
*Optional: write it only when the task touches a guard, a gate, untrusted input,
|
||||
secrets or publishing, and leave the whole section out otherwise (absent = every
|
||||
security finding counts). One line per bullet, each `- Inside: <text>` (what the
|
||||
work must defend against) or `- Outside: <text>` (what it deliberately does not);
|
||||
any other bullet makes the brief invalid (`BRIEF_THREAT_MODEL_MALFORMED`).
|
||||
Voyage's own security findings block inside it; one that quotes an `Outside:`
|
||||
line's text exactly becomes a listed known issue. The intent approval covers
|
||||
this section: editing it afterwards makes the approval stale.*
|
||||
|
||||
- Inside: {e.g., "a crafted file name in a pull request reaches the shell"}
|
||||
- Outside: {e.g., "a local user who can already edit the repo's hooks"}
|
||||
|
||||
## Success Criteria
|
||||
|
||||
*Falsifiable, command-checkable conditions that define "done". Each must be
|
||||
|
|
|
|||
|
|
@ -129,6 +129,7 @@ const IN_SCOPE_CODES = [
|
|||
'BRIEF_MISSING_SECTION',
|
||||
'BRIEF_INVALID_FRAMING',
|
||||
'BRIEF_MISSING_FRAMING',
|
||||
'BRIEF_THREAT_MODEL_MALFORMED',
|
||||
];
|
||||
|
||||
const CASES = {
|
||||
|
|
@ -164,6 +165,10 @@ const CASES = {
|
|||
fail: GOOD_BRIEF_22.replace('framing: new-direction\n', ''),
|
||||
pass: GOOD_BRIEF_22,
|
||||
},
|
||||
BRIEF_THREAT_MODEL_MALFORMED: {
|
||||
fail: GOOD_BRIEF_22.replace('## Success Criteria', '## Threat Model\n\n- Scope: the API\n\n## Success Criteria'),
|
||||
pass: GOOD_BRIEF_22.replace('## Success Criteria', '## Threat Model\n\n- Inside: a forged token\n- Outside: a stolen signing key\n\n## Success Criteria'),
|
||||
},
|
||||
};
|
||||
|
||||
test('gate coverage — CASES keys exactly match IN_SCOPE_CODES (no silent gap)', () => {
|
||||
|
|
|
|||
214
tests/validators/brief-threat-model.test.mjs
Normal file
214
tests/validators/brief-threat-model.test.mjs
Normal file
|
|
@ -0,0 +1,214 @@
|
|||
// tests/validators/brief-threat-model.test.mjs
|
||||
// E 2.1 — the optional `## Threat Model` in a brief (decisions 164, 218):
|
||||
// bullets `- Inside: <text>` / `- Outside: <text>`. Present and malformed (no
|
||||
// bullet, another label, empty text, two such sections) is
|
||||
// BRIEF_THREAT_MODEL_MALFORMED; absent is today's rule. No brief_version bump
|
||||
// (the ceremony / artifact_store precedent). The intent hash covers the section
|
||||
// when it is present, so an edited threat model makes the approval stale; a
|
||||
// brief without one hashes as before.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = join(HERE, '..', '..');
|
||||
const validator = () => import(join(ROOT, 'lib', 'validators', 'brief-validator.mjs'));
|
||||
const approval = () => import(join(ROOT, 'lib', 'validators', 'intent-approval.mjs'));
|
||||
const read = (rel) => readFileSync(join(ROOT, rel), 'utf8');
|
||||
|
||||
const TM = `## Threat Model
|
||||
|
||||
- Inside: a crafted file name in a pull request reaches the shell
|
||||
- Outside: a local user who can already edit the repo's hooks
|
||||
`;
|
||||
|
||||
const brief = (threat = '') => `---
|
||||
type: trekbrief
|
||||
brief_version: "2.2"
|
||||
created: 2026-11-08
|
||||
task: "Add an upload hook"
|
||||
slug: upload-hook
|
||||
project_dir: .claude/projects/2026-11-08-upload-hook/
|
||||
research_topics: 0
|
||||
research_status: complete
|
||||
framing: new-direction
|
||||
---
|
||||
|
||||
# Task
|
||||
|
||||
## TL;DR
|
||||
|
||||
An upload hook.
|
||||
|
||||
## Intent
|
||||
|
||||
Uploads arrive from strangers.
|
||||
|
||||
## Goal
|
||||
|
||||
A hook that checks each upload.
|
||||
|
||||
${threat}
|
||||
## Success Criteria
|
||||
|
||||
- It checks.
|
||||
`;
|
||||
|
||||
const codes = (r) => r.errors.map((e) => e.code);
|
||||
const STAMP_AT = new Date('2026-11-08T08:00:00Z');
|
||||
const FENCE = '```';
|
||||
|
||||
// ---- the validator ----
|
||||
|
||||
test('a well-formed threat model is valid; absent is today\'s rule', async () => {
|
||||
const { validateBriefContent } = await validator();
|
||||
assert.deepEqual(codes(validateBriefContent(brief(TM))), []);
|
||||
assert.deepEqual(codes(validateBriefContent(brief())), []);
|
||||
const onlyInside = '## Threat Model\n\n- Inside: untrusted input from a web form\n';
|
||||
assert.deepEqual(codes(validateBriefContent(brief(onlyInside))), []);
|
||||
});
|
||||
|
||||
test('malformed: no bullet, another label, empty text, two sections — each BRIEF_THREAT_MODEL_MALFORMED', async () => {
|
||||
const { validateBriefContent } = await validator();
|
||||
const cases = {
|
||||
'no bullet': '## Threat Model\n\nAnything a stranger sends.\n',
|
||||
'empty section': '## Threat Model\n\n',
|
||||
'another label': '## Threat Model\n\n- Inside: uploads\n- Scope: the whole app\n',
|
||||
'lower-case label': '## Threat Model\n\n- outside: a local user\n',
|
||||
'empty text': '## Threat Model\n\n- Inside: uploads\n- Outside:\n',
|
||||
'blank text': '## Threat Model\n\n- Inside: \n',
|
||||
'two sections': `${TM}\n${TM}`,
|
||||
};
|
||||
for (const [name, section] of Object.entries(cases)) {
|
||||
const r = validateBriefContent(brief(section));
|
||||
assert.equal(r.valid, false, `${name}: must not be valid`);
|
||||
assert.deepEqual(codes(r), ['BRIEF_THREAT_MODEL_MALFORMED'], name);
|
||||
}
|
||||
});
|
||||
|
||||
test('a ## Threat Model inside a code fence is text, not a section (no false malformed, no second section)', async () => {
|
||||
const { validateBriefContent } = await validator();
|
||||
const fenced = `${FENCE}markdown\n## Threat Model\n\nnot a bullet\n${FENCE}\n`;
|
||||
assert.deepEqual(codes(validateBriefContent(brief(fenced))), []);
|
||||
assert.deepEqual(codes(validateBriefContent(brief(`${TM}\n${fenced}`))), []);
|
||||
});
|
||||
|
||||
test('the malformed message names the section and the bullet form', async () => {
|
||||
const { validateBriefContent } = await validator();
|
||||
const e = validateBriefContent(brief('## Threat Model\n\n- Scope: all\n')).errors[0];
|
||||
assert.ok(e, 'an error');
|
||||
assert.match(`${e.message} ${e.hint || ''}`, /## Threat Model/);
|
||||
assert.match(`${e.message} ${e.hint || ''}`, /- Inside: <text>/);
|
||||
assert.match(`${e.message} ${e.hint || ''}`, /- Outside: <text>/);
|
||||
});
|
||||
|
||||
test('no brief_version bump: the template stays at 2.2 and documents the optional section with both bullets', () => {
|
||||
const tpl = read('templates/trekbrief-template.md');
|
||||
assert.match(tpl, /^brief_version: "2\.2"$/m);
|
||||
const at = tpl.indexOf('\n## Threat Model');
|
||||
assert.ok(at > 0, 'the template documents ## Threat Model');
|
||||
const sec = tpl.slice(at, tpl.indexOf('\n## ', at + 4));
|
||||
assert.match(sec, /optional/i);
|
||||
assert.match(sec, /^- Inside: /m);
|
||||
assert.match(sec, /^- Outside: /m);
|
||||
});
|
||||
|
||||
// ---- the intent hash ----
|
||||
|
||||
test('red before E: an edited threat model makes the approval stale (BRIEF_INTENT_APPROVAL_STALE)', async () => {
|
||||
const { stampIntentApproval, checkIntentApprovalContent } = await approval();
|
||||
const stamped = stampIntentApproval(brief(TM), STAMP_AT).text;
|
||||
assert.deepEqual(codes(checkIntentApprovalContent(stamped)), []);
|
||||
const edited = stamped.replace("can already edit the repo's hooks", 'can reach the network');
|
||||
assert.notEqual(edited, stamped);
|
||||
assert.deepEqual(codes(checkIntentApprovalContent(edited)), ['BRIEF_INTENT_APPROVAL_STALE']);
|
||||
});
|
||||
|
||||
test('adding or removing a threat model after the approval makes it stale', async () => {
|
||||
const { stampIntentApproval, checkIntentApprovalContent } = await approval();
|
||||
const plain = stampIntentApproval(brief(), STAMP_AT).text;
|
||||
const added = plain.replace('## Success Criteria', `${TM}\n## Success Criteria`);
|
||||
assert.deepEqual(codes(checkIntentApprovalContent(added)), ['BRIEF_INTENT_APPROVAL_STALE'], 'added');
|
||||
const withTm = stampIntentApproval(brief(TM), STAMP_AT).text;
|
||||
const removed = withTm.replace(TM, '');
|
||||
assert.deepEqual(codes(checkIntentApprovalContent(removed)), ['BRIEF_INTENT_APPROVAL_STALE'], 'removed');
|
||||
});
|
||||
|
||||
test('two ## Threat Model sections: null hash, the stamp refuses and the check is INVALID', async () => {
|
||||
const { computeIntentHash, stampIntentApproval, checkIntentApprovalContent } = await approval();
|
||||
const two = brief(`${TM}\n${TM}`);
|
||||
const body = two.split('---\n').slice(2).join('---\n');
|
||||
const h = computeIntentHash(body);
|
||||
assert.equal(h.hash, null);
|
||||
assert.deepEqual(h.duplicate, ['Threat Model']);
|
||||
assert.equal(stampIntentApproval(two).stamped, false);
|
||||
const forged = two.replace('framing: new-direction\n',
|
||||
`framing: new-direction\nintent_approved_hash: "sha256:${'0'.repeat(64)}"\n`);
|
||||
const r = checkIntentApprovalContent(forged);
|
||||
assert.deepEqual(codes(r), ['BRIEF_INTENT_APPROVAL_INVALID']);
|
||||
assert.match(r.errors[0].message, /## Threat Model appears more than once/);
|
||||
});
|
||||
|
||||
test('whitespace-only reflow inside the threat model keeps the approval (normalized, as Intent and Goal)', async () => {
|
||||
const { stampIntentApproval, checkIntentApprovalContent } = await approval();
|
||||
const stamped = stampIntentApproval(brief(TM), STAMP_AT).text;
|
||||
const reflowed = stamped.replace('- Inside: a crafted file name', '- Inside: a crafted file name');
|
||||
assert.notEqual(reflowed, stamped);
|
||||
assert.deepEqual(codes(checkIntentApprovalContent(reflowed)), []);
|
||||
});
|
||||
|
||||
test('a brief without a threat model hashes exactly as before (no stamped brief goes stale)', async () => {
|
||||
const { computeIntentHash } = await approval();
|
||||
const { createHash } = await import('node:crypto');
|
||||
const body = brief().split('---\n').slice(2).join('---\n');
|
||||
const parts = ['Uploads arrive from strangers.', 'A hook that checks each upload.'];
|
||||
const expected = `sha256:${createHash('sha256').update(JSON.stringify(parts), 'utf8').digest('hex')}`;
|
||||
assert.equal(computeIntentHash(body).hash, expected);
|
||||
});
|
||||
|
||||
test('the stale message names ## Threat Model (intent-approval.mjs and the /trekplan halt table)', async () => {
|
||||
const { stampIntentApproval, checkIntentApprovalContent } = await approval();
|
||||
const stamped = stampIntentApproval(brief(TM), STAMP_AT).text;
|
||||
const e = checkIntentApprovalContent(stamped.replace('reaches the shell', 'reaches a log')).errors[0];
|
||||
assert.ok(e, 'a stale error');
|
||||
assert.match(e.message, /## Threat Model/);
|
||||
const plan = read('commands/trekplan.md');
|
||||
const row = plan.split('\n').find((l) => l.startsWith('| `BRIEF_INTENT_APPROVAL_STALE` |'));
|
||||
assert.ok(row, 'the halt table has the stale row');
|
||||
assert.match(row, /`## Threat Model`/);
|
||||
});
|
||||
|
||||
// ---- /trekbrief prose ----
|
||||
|
||||
function section(text, heading) {
|
||||
const start = text.indexOf(`\n## ${heading}`);
|
||||
assert.ok(start >= 0, `missing ## ${heading}`);
|
||||
const next = text.indexOf('\n## ', start + 4);
|
||||
return text.slice(start, next < 0 ? undefined : next);
|
||||
}
|
||||
|
||||
test('/trekbrief offers the section when the task touches a guard, a gate, untrusted input, secrets or publishing', () => {
|
||||
const p3 = section(read('commands/trekbrief.md'), 'Phase 3').replace(/\s+/g, ' ');
|
||||
assert.match(p3, /`## Threat Model`/);
|
||||
for (const w of ['a guard', 'a gate', 'untrusted input', 'secrets', 'publishing']) {
|
||||
assert.ok(p3.includes(w), `Phase 3 names «${w}» as a reason to offer the threat model`);
|
||||
}
|
||||
});
|
||||
|
||||
test('/trekbrief writes the section only when given, as one-line Inside/Outside bullets', () => {
|
||||
const text = read('commands/trekbrief.md');
|
||||
const draft = text.slice(text.indexOf('**Step 4a'), text.indexOf('**Step 4b')).replace(/\s+/g, ' ');
|
||||
assert.match(draft, /\*\*Threat Model:\*\*/);
|
||||
assert.match(draft, /`- Inside: <text>`/);
|
||||
assert.match(draft, /`- Outside: <text>`/);
|
||||
});
|
||||
|
||||
test('/trekbrief Phase 4h shows ## Threat Model verbatim at the approval, and --approve reads it out', () => {
|
||||
const p4h = section(read('commands/trekbrief.md'), 'Phase 4h').replace(/\s+/g, ' ');
|
||||
assert.match(p4h, /`## Intent`, `## Goal` and, when present, `## Threat Model`[^.]*\*\*verbatim\*\*/);
|
||||
assert.match(p4h, /`--approve` mode[^.]*`## Threat Model`/);
|
||||
assert.match(p4h, /`## Threat Model`[^.]*afterwards makes the approval stale/);
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue