feat(llm-security)!: v8 Phase 3 complete - riskScoreV1, posture heuristic, docs

Closes Phase 3 (B11) of the v8.0.0 plan. Three parts, all with the failing
test written first.

riskScoreV1 removed. scanners/lib/severity.mjs drops riskScoreV1() and its
SEVERITY_WEIGHTS_V1 table - @deprecated since v7.0.0, kept for diff/comparison,
zero callers in code or tests (re-verified, not taken from the plan). The v1
weights are recorded in CHANGELOG so an old score stays re-derivable. riskScore
(v2) is untouched; a test pins that one critical still lands in the 70-95 tier
and that 50 lows score below it, which is exactly the case v1 collapsed to 100.

Posture category 12 no longer keys off an identifier name. The check was
/TRIFECTA_MODE/i over the session-guard source, which measured what a constant
was CALLED rather than whether enforcement was configurable. With the env-var
gone, that regex would have dropped every correctly-migrated project from PASS
to PARTIAL - the gate punishing the migration it exists to encourage. It now
matches getPolicyValue('trifecta', 'mode', ...) and still accepts a pre-v8
vendored guard reading the old env-var, because a third-party project carries
its own hook copy and is equally configurable either way; the evidence line
says which of the two was found. The PARTIAL finding recommended setting an
env-var that v8 ignores; it now names the policy key. The grade-a fixture hook
moves to the policy-era form.

Two never-implemented env-vars deleted from the docs. LLM_SECURITY_SCR_OFFLINE
(ci-cd-guide) and LLM_SECURITY_OFFLINE (supply-chain-attack example) were
documented as OSV.dev / npm-audit kill-switches. No code has ever read either -
verified by grep across scanners, hooks and scripts, which finds them only in
markdown. A promised kill-switch that does nothing is worse than a documented
absence: it is trusted precisely when the run is meant to be air-gapped. The
docs now say there is none and that egress must be blocked at the network
layer. The LLM_SECURITY_AUDIT_* wildcard is narrowed to the one real key.

Docs. Migration section in README + CHANGELOG with the env-var -> policy-key
table, the detection commands (env + shell rc + .envrc + workflows), and the
explicit warning that a removed variable is now INERT rather than an error -
which is the failure mode that loses a project its configuration silently. The
hardening-guide env table splits into surviving vars and a removed-vars
migration table; its "promote to block" runbook named two variables that no
longer exist. Also swept: CLAUDE.md hook table, scanner-reference, ci-cd-guide,
both lethal-trifecta example docs, mitigation-matrix, injection-research.

Test counts in README/CLAUDE.md synced 2034 -> 2045.

Suite 2045 tests, 0 fail (2039 + 4 posture-trifecta + 2 riskScoreV1). The two
known parallel-load flakes did not recur this run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 10:25:03 +02:00
commit fdec4b36ad
16 changed files with 333 additions and 62 deletions

View file

@ -6,6 +6,61 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]
### Removed — BREAKING
- **The four `LLM_SECURITY_*` configuration env-vars deprecated in v7.3.0.**
`.llm-security/policy.json` is now the only source. A removed variable is
**inert**: it neither warns nor configures, so a project relying on one
silently returns to the default. See the Migration section in `README.md`
for the detection commands.
| Removed env-var | Policy key | Default |
|-----------------|------------|---------|
| `LLM_SECURITY_INJECTION_MODE` | `injection.mode` | `block` |
| `LLM_SECURITY_TRIFECTA_MODE` | `trifecta.mode` | `warn` |
| `LLM_SECURITY_ESCALATION_WINDOW` | `trifecta.escalation_window` | `5` |
| `LLM_SECURITY_AUDIT_LOG` | `audit.log_path` | unset |
| `LLM_SECURITY_DEPRECATION_QUIET` | *(none)* | dies with the warning it silenced |
Env-vars with no policy equivalent are unaffected:
`LLM_SECURITY_PRECOMPACT_MODE`, `LLM_SECURITY_PRECOMPACT_MAX_BYTES`,
`LLM_SECURITY_UPDATE_CHECK`, `LLM_SECURITY_MCP_CACHE_FILE`,
`LLM_SECURITY_IDE_ROOTS`.
`getPolicyValueWithEnvWarn` and its one-shot stderr deprecation line are
deleted from `scanners/lib/policy-loader.mjs`; the four call sites collapse
to `getPolicyValue`. Hook strings that advertised a removed variable as the
escape hatch now name the policy key — a blocked user following the old text
would have set a variable that does nothing.
- **`riskScoreV1()`** in `scanners/lib/severity.mjs`, plus its
`SEVERITY_WEIGHTS_V1` table. `@deprecated` since v7.0.0, kept for
diff/comparison, with zero callers in code or tests. The v1 weights are
recorded here for anyone re-deriving an old score:
`critical 25, high 10, medium 4, low 1, info 0`, summed and capped at 100.
`riskScore()` (v2) is untouched, so no score, band, or verdict moves.
- **Two documented env-vars that were never implemented.**
`LLM_SECURITY_SCR_OFFLINE` (`docs/ci-cd-guide.md`) and `LLM_SECURITY_OFFLINE`
(`examples/supply-chain-attack/README.md`) were promised as OSV.dev / npm-audit
kill-switches; no code has ever read either. The docs now say plainly that
there is no kill-switch and that an air-gapped run must block egress at the
network layer. The `LLM_SECURITY_AUDIT_*` wildcard phrasing (README,
`docs/scanner-reference.md`, `docs/security-hardening-guide.md`) is narrowed
to the single real key.
### Changed
- **Posture category 12 (Rule of Two) no longer keys off the identifier
`TRIFECTA_MODE`.** The check was `/TRIFECTA_MODE/i` over the session-guard
source, which measured what a constant was *named* rather than whether
enforcement was configurable at all. It now matches
`getPolicyValue('trifecta', 'mode', …)`, and still accepts a pre-v8 vendored
guard reading `LLM_SECURITY_TRIFECTA_MODE` — third-party projects carry their
own hook copy and are equally configurable. Without this, every correctly
migrated project would have dropped from PASS to PARTIAL. The PARTIAL
finding now recommends the policy key instead of the removed env-var.
## [7.8.3] - 2026-07-18
Security and correctness patch. 47 verified fixes from the v7.8.1/v7.8.2

View file

@ -1,6 +1,6 @@
# LLM Security Plugin (v7.8.3)
Security scanning, auditing, and threat modeling for Claude Code projects. 5 frameworks: OWASP LLM Top 10, Agentic AI Top 10 (ASI, 2026 edition), Skills Top 10 (AST), MCP Top 10, AI Agent Traps (DeepMind). 2034+ unit, integration, and end-to-end tests (`tests/e2e/` covers the multi-hook attack chain, multi-session state simulation, and the full scan-orchestrator pipeline); mutation-testing coverage not published.
Security scanning, auditing, and threat modeling for Claude Code projects. 5 frameworks: OWASP LLM Top 10, Agentic AI Top 10 (ASI, 2026 edition), Skills Top 10 (AST), MCP Top 10, AI Agent Traps (DeepMind). 2045+ unit, integration, and end-to-end tests (`tests/e2e/` covers the multi-hook attack chain, multi-session state simulation, and the full scan-orchestrator pipeline); mutation-testing coverage not published.
Release notes for v7.0.0 → v7.8.2: see `docs/version-history.md` — read on demand.
@ -58,13 +58,13 @@ Release notes for v7.0.0 → v7.8.2: see `docs/version-history.md` — read on d
| Script | Event | Matcher | Purpose |
|--------|-------|---------|---------|
| `pre-prompt-inject-scan.mjs` | UserPromptSubmit | — | Block prompt injection, warn on manipulation (incl. oversight evasion, HTML obfuscation, MEDIUM advisory for leetspeak/homoglyphs/zero-width/multi-lang). Unicode Tag steganography detection. Mode: `LLM_SECURITY_INJECTION_MODE=block\|warn\|off` |
| `pre-prompt-inject-scan.mjs` | UserPromptSubmit | — | Block prompt injection, warn on manipulation (incl. oversight evasion, HTML obfuscation, MEDIUM advisory for leetspeak/homoglyphs/zero-width/multi-lang). Unicode Tag steganography detection. Mode: policy key `injection.mode` = `block\|warn\|off` |
| `pre-edit-secrets.mjs` | PreToolUse | `Edit\|Write` | Block credentials in files |
| `pre-bash-destructive.mjs` | PreToolUse | `Bash` | Block rm -rf, curl\|sh, fork bombs, eval. Bash evasion normalization (T1-T6 via `bash-normalize.mjs`) — defense-in-depth |
| `pre-install-supply-chain.mjs` | PreToolUse | `Bash` | Block compromised packages across ALL ecosystems. Bash evasion normalization before gate matching |
| `pre-write-pathguard.mjs` | PreToolUse | `Edit\|Write` | Block writes to .env, .ssh/, .aws/, credentials, settings |
| `post-mcp-verify.mjs` | PostToolUse | — (all) | Injection scan on ALL tool output. MCP per-update drift + cumulative drift vs sticky baseline (E14, v7.3.0). Per-tool volume tracking |
| `post-session-guard.mjs` | PostToolUse | — (all) | Runtime trifecta detection (Rule of Two). Sliding window + long-horizon. Behavioral drift (Jensen-Shannon). Mode: `LLM_SECURITY_TRIFECTA_MODE=block\|warn\|off` (default: warn) |
| `post-session-guard.mjs` | PostToolUse | — (all) | Runtime trifecta detection (Rule of Two). Sliding window + long-horizon. Behavioral drift (Jensen-Shannon). Mode: policy key `trifecta.mode` = `block\|warn\|off` (default: warn) |
| `update-check.mjs` | UserPromptSubmit | — | Checks for newer versions (max 1x/24h, cached). Disable: `LLM_SECURITY_UPDATE_CHECK=off` |
| `pre-compact-scan.mjs` | PreCompact | — | Scan transcript for injection + credentials before context compaction. Reads at most last 512 KB. Mode: `LLM_SECURITY_PRECOMPACT_MODE=block\|warn\|off` (default: warn) |

View file

@ -199,13 +199,13 @@ Hooks run on every operation — no commands needed. They activate the moment th
| Hook | Event | What it does |
|------|-------|--------------|
| **Prompt injection scan** | UserPromptSubmit | Blocks direct injection (override instructions, spoofed system headers, identity redefinition) and warns on subtle signals (leetspeak, homoglyphs, zero-width chars, multi-language). Decodes obfuscated payloads (Unicode Tag, hex, URL, base64, rot13) before matching. Mode: `LLM_SECURITY_INJECTION_MODE=block\|warn\|off` (default block) |
| **Prompt injection scan** | UserPromptSubmit | Blocks direct injection (override instructions, spoofed system headers, identity redefinition) and warns on subtle signals (leetspeak, homoglyphs, zero-width chars, multi-language). Decodes obfuscated payloads (Unicode Tag, hex, URL, base64, rot13) before matching. Mode: policy key `injection.mode` = `block\|warn\|off` (default block) |
| **Secret detection** | Edit, Write | Blocks AWS keys, Azure tokens, GitHub PATs, npm tokens, PEM keys, database URLs, Bearer tokens, and 30+ other secret patterns |
| **Path guarding** | Write | Blocks writes to `.env*` (multi-segment-suffix-safe), `.ssh/`, `.aws/`, `.gnupg/`, credentials files, hook scripts, `/etc/`, `settings.json` |
| **Destructive commands** | Bash | Blocks `rm -rf /`, `chmod 777`, pipe-to-shell, fork bombs, eval-with-substitution, T8 base64-pipe-shell loaders. Bash-normalize T1-T9 collapses obfuscation (empty quotes, `${IFS}`, ANSI-C hex, process substitution, eval-via-variable) before pattern matching |
| **Supply-chain guardrail** | Bash | Blocks known-compromised npm/pip packages, Levenshtein typosquats, age-gated installs (<72 h), OSV.dev CVE checks. Covers npm, pip, brew, docker, go, cargo, gem. v7.3.0: npm scope-hop typosquat advisory (E13) — `@evil/lodash`-class catches scope-jumping when the unscoped name matches a popular package |
| **Output verification** | All tools (post) | Advisory: scans ALL tool output for indirect injection (LLM01) and HITL traps (DeepMind kat. 6). Bash-specific: leaked secrets, unexpected URLs, oversized MCP responses. v7.3.0: per-update MCP description drift AND cumulative drift vs sticky baseline (E14) — slow-burn rug-pulls that stay under per-update thresholds but cumulatively diverge ≥25% emit `mcp-cumulative-drift` MEDIUM |
| **Session guard** | All tools (post) | Advisory: monitors tool-call sequences for the lethal trifecta (untrusted input + sensitive read + exfiltration sink). 20-call sliding window + 100-call long-horizon window. Mode: `LLM_SECURITY_TRIFECTA_MODE=block\|warn\|off`. Sub-agent delegation tracking via Task/Agent tools surfaces escalation-after-input as a separate advisory |
| **Session guard** | All tools (post) | Advisory: monitors tool-call sequences for the lethal trifecta (untrusted input + sensitive read + exfiltration sink). 20-call sliding window + 100-call long-horizon window. Mode: policy key `trifecta.mode` = `block\|warn\|off`. Sub-agent delegation tracking via Task/Agent tools surfaces escalation-after-input as a separate advisory |
| **Pre-compact scan** | PreCompact | Scans transcript tail (max 512 KB, <500 ms) for injection patterns + credentials before context compaction. Prevents poisoned content from surviving in compact form. Mode: `LLM_SECURITY_PRECOMPACT_MODE=block\|warn\|off` (default warn) |
| **Update check** | UserPromptSubmit | Checks for newer plugin versions max 1× / 24 h, cached. Disable: `LLM_SECURITY_UPDATE_CHECK=off` |
@ -329,9 +329,9 @@ Average ~69 %. Strongest at prompt injection (95 % with input + output scanning
| **Compliance mapping** | EU AI Act (Art. 9, 15, 17), NIST AI RMF (Map / Measure / Manage / Govern), ISO 42001 (Annex A), MITRE ATLAS techniques. Posture categories 14-16 assess readiness |
| **Norwegian context** | Datatilsynet DPIA-for-AI guidance, NSM basic security principles, Digitaliseringsdirektoratet — relevant for Norwegian public-sector deployments |
| **SARIF 2.1.0 output** | `--format sarif` on scan / deep-scan produces OASIS SARIF for CI/CD ingestion (GitHub Advanced Security, Azure DevOps, SonarQube) |
| **Structured audit trail** | JSONL events with ISO 8601 timestamps and OWASP category tags (`LLM_SECURITY_AUDIT_*` env-vars or `audit.log_path` policy key) — SIEM-ready |
| **Structured audit trail** | JSONL events with ISO 8601 timestamps and OWASP category tags (`audit.log_path` policy key) — SIEM-ready |
| **AI-BOM** | CycloneDX 1.6 BOM for AI components — models, MCP servers, plugins, knowledge files, hooks (`llm-security audit-bom <target>`) |
| **Policy-as-code** | `.llm-security/policy.json` ships hook configuration with the team. v7.3.0 (D3) adds a one-time-per-process stderr deprecation line when both an env-var AND its `policy.json` equivalent are explicitly set; env still wins through the v7.x runway, env reads removed in v8.0.0. Suppress noise with `LLM_SECURITY_DEPRECATION_QUIET=1` |
| **Policy-as-code** | `.llm-security/policy.json` ships hook configuration with the team — the only source for injection, trifecta, and audit configuration since v8.0.0 removed the overlapping env-vars |
| **Standalone CLI** | `node bin/llm-security.mjs scan <target>` — runs scanners without Claude Code. Subcommands: `scan`, `deep-scan`, `posture`, `audit-bom`, `benchmark`. Schrems II compatible in default offline mode (optional OSV.dev enrichment is the only network call and is opt-in) |
| **CI/CD integration** | `--fail-on <severity>` for threshold-based exit codes, `--compact` for one-liner output. Templates for GitHub Actions, Azure DevOps, GitLab CI in `ci/`. Guide: `docs/ci-cd-guide.md` |
@ -419,14 +419,14 @@ These gaps are surfaced advisorily through `/security threat-model` and `/securi
This is a **solo open-source project in stabilization mode** as of 2026-05-01.
The current feature set (5 frameworks, 22 scanners, 9 hooks, 6 agents,
20 commands, 23 knowledge files, 2034+ tests including a dedicated end-to-end suite) is the natural plateau for
20 commands, 23 knowledge files, 2045+ tests including a dedicated end-to-end suite) is the natural plateau for
what a deterministic + advisory plugin can defend against without crossing
into commercial-grade territory. Going forward, work focuses on:
- **Bug fixes** and security patches
- **Compatibility** with new Claude Code releases
- **Knowledge-base refresh** (OWASP updates, new published research, new attack patterns)
- **Deprecation cleanup** — v8.0.0 removes the `LLM_SECURITY_*` env vars and `riskScoreV1` constant deprecated in v7.3.0
- **Deprecation cleanup** — v8.0.0 removed the four `LLM_SECURITY_*` mode env vars and `riskScoreV1`, both deprecated in v7.3.0 (see [Migration](#migrating-to-v800))
- **Opportunistic small additions** that fit the existing deterministic architecture
## Non-goals
@ -479,6 +479,53 @@ Prompt injection is **structurally unsolvable** with current architectures (join
---
## Migrating to v8.0.0
v8.0.0 removes the four `LLM_SECURITY_*` configuration env-vars deprecated in
v7.3.0. Configuration moved to `.llm-security/policy.json`, which travels with
the repository instead of living in whoever's shell happened to launch Claude
Code.
**A removed variable is now inert.** It does not warn and it does not
configure — a project that relied on `LLM_SECURITY_INJECTION_MODE=off` silently
returns to the `block` default. Check for these before upgrading:
```bash
env | grep '^LLM_SECURITY_\(INJECTION_MODE\|TRIFECTA_MODE\|ESCALATION_WINDOW\|AUDIT_LOG\|DEPRECATION_QUIET\)='
grep -rn 'LLM_SECURITY_\(INJECTION_MODE\|TRIFECTA_MODE\|ESCALATION_WINDOW\|AUDIT_LOG\)' \
~/.zshenv ~/.bashrc .envrc .github/workflows/ 2>/dev/null
```
| Removed env-var | Policy key | Default |
|-----------------|------------|---------|
| `LLM_SECURITY_INJECTION_MODE` | `injection.mode` | `block` |
| `LLM_SECURITY_TRIFECTA_MODE` | `trifecta.mode` | `warn` |
| `LLM_SECURITY_ESCALATION_WINDOW` | `trifecta.escalation_window` | `5` |
| `LLM_SECURITY_AUDIT_LOG` | `audit.log_path` | unset (audit trail off) |
| `LLM_SECURITY_DEPRECATION_QUIET` | *(none)* | removed with the warning it silenced |
Translate each one you find into `.llm-security/policy.json`:
```json
{
"injection": { "mode": "block" },
"trifecta": { "mode": "warn", "escalation_window": 5 },
"audit": { "log_path": "/var/log/llm-security/audit.jsonl" }
}
```
**Unaffected.** Env-vars with no policy equivalent keep working and are not
part of this change: `LLM_SECURITY_PRECOMPACT_MODE`,
`LLM_SECURITY_PRECOMPACT_MAX_BYTES`, `LLM_SECURITY_UPDATE_CHECK`,
`LLM_SECURITY_MCP_CACHE_FILE`, `LLM_SECURITY_IDE_ROOTS`.
**Also removed:** `riskScoreV1()` in `scanners/lib/severity.mjs`, the v1
sum-and-cap scoring formula `@deprecated` since v7.0.0 and kept for reference
only. It had no callers in code or tests. `riskScore()` (v2, severity-dominated)
is unchanged, so no score, band, or verdict moves.
---
## Playground (v7.6.0)
A single-file SPA at `playground/llm-security-playground.html` provides

View file

@ -6,7 +6,7 @@ Integrate llm-security into your CI/CD pipeline for automated security scanning
**The standalone CLI makes zero network calls by default.** All 14 scanners operate locally on your source code using Shannon entropy analysis, regex pattern matching, AST traversal, and git log parsing. No data is transmitted to any external service.
**Exception: supply-chain-recheck** — When scanning lockfiles for known vulnerabilities, this scanner optionally queries the [OSV.dev](https://osv.dev/) batch API. This sends only package names and versions (not source code) over HTTPS. To disable: set `LLM_SECURITY_SCR_OFFLINE=1`.
**Exception: supply-chain-recheck** — When scanning lockfiles for known vulnerabilities, this scanner optionally queries the [OSV.dev](https://osv.dev/) batch API. This sends only package names and versions (not source code) over HTTPS. There is no kill-switch for this call today; block the host at the network layer if the run must be air-gapped.
**What about Claude Code integration?** The Claude Code plugin (hooks, agents, commands) uses AI models and sends data to Anthropic. These components are **not included** in the standalone CLI. When you run `npx llm-security scan`, only deterministic scanners execute.
@ -122,8 +122,11 @@ CLI flags always take precedence over policy file values.
| Variable | Description |
|----------|-------------|
| `LLM_SECURITY_SCR_OFFLINE=1` | Disable OSV.dev network calls in supply-chain-recheck |
| `LLM_SECURITY_AUDIT_LOG=<path>` | Write structured JSONL audit trail (SIEM-ready) |
| `LLM_SECURITY_PRECOMPACT_MODE` | `block\|warn\|off` for the PreCompact transcript scan (default `warn`) |
| `LLM_SECURITY_UPDATE_CHECK=off` | Disable the daily update-check HTTP call |
The audit trail moved to the policy file in v8.0.0: set `audit.log_path` in
`.llm-security/policy.json` instead of `LLM_SECURITY_AUDIT_LOG`.
## Exit Codes

View file

@ -27,7 +27,7 @@ AST-taint (AST) shells out to a PARSE-ONLY python3 helper (`scanners/lib/py-ast-
Utility: `node scanners/lib/fs-utils.mjs <backup|restore|cleanup|tmppath> [args]`.
Lib: `sarif-formatter.mjs` — converts scan output to OASIS SARIF 2.1.0 format. Used by `--format sarif` flag.
Lib: `audit-trail.mjs` — writes structured JSONL audit events (ISO 8601, OWASP tags, SIEM-ready). Env: `LLM_SECURITY_AUDIT_*`.
Lib: `audit-trail.mjs` — writes structured JSONL audit events (ISO 8601, OWASP tags, SIEM-ready). Enabled by the `audit.log_path` key in `.llm-security/policy.json` (v8.0.0; was `LLM_SECURITY_AUDIT_LOG`).
Lib: `policy-loader.mjs` — reads `.llm-security/policy.json` for distributable hook configuration. Includes `ci` section (`failOn`, `compact`) for CI/CD defaults. Defaults match hardcoded values.
**Standalone (8):** `posture-scanner.mjs` — deterministic posture assessment, 16 categories (incl. EU AI Act, NIST AI RMF, ISO 42001), <50ms. NOT in scan-orchestrator (meta-level, not code-level).

View file

@ -23,16 +23,33 @@ in production. Deviations are fine, but the defaults here are the tested path.
| Variable | Default | Modes |
|----------|---------|-------|
| `LLM_SECURITY_INJECTION_MODE` | `block` | `block` — exit 2 on critical/high injection patterns. `warn` — advisory via systemMessage. `off` — disables scan. |
| `LLM_SECURITY_TRIFECTA_MODE` | `warn` | `block` — exit 2 when lethal trifecta (untrusted input + sensitive data + exfiltration sink) detected. `warn` — advisory. `off` — disables. |
| `LLM_SECURITY_PRECOMPACT_MODE` | `warn` | `block` — exit 2 on findings during PreCompact. `warn` — advisory via systemMessage. `off` — disables scan. |
| `LLM_SECURITY_PRECOMPACT_MAX_BYTES` | `512000` | Tail size in bytes read from transcript for scanning. Higher values increase coverage at the cost of latency. |
| `LLM_SECURITY_UPDATE_CHECK` | `on` | `off` disables the daily update-check HTTP call. |
| `LLM_SECURITY_AUDIT_*` | unset | Audit trail configuration (destination, format, etc.) for SIEM-ready JSONL output. |
Apply env vars via shell profile, `.envrc`, or the host MDM. Do not write them
into the repository.
**Removed in v8.0.0.** Four modes moved to `.llm-security/policy.json`, which
travels with the repository instead of the operator's shell. Setting the old
variable is now inert — it neither warns nor configures.
| Removed variable | Policy key | Default |
|------------------|-----------|---------|
| `LLM_SECURITY_INJECTION_MODE` | `injection.mode` | `block` — exit 2 on critical injection patterns. `warn` — advisory via systemMessage. `off` — disables scan. |
| `LLM_SECURITY_TRIFECTA_MODE` | `trifecta.mode` | `warn` — advisory. `block` — exit 2 when the lethal trifecta (untrusted input + sensitive data + exfiltration sink) is detected. `off` — disables. |
| `LLM_SECURITY_ESCALATION_WINDOW` | `trifecta.escalation_window` | `5` — calls between untrusted input and sub-agent delegation that still count as escalation-after-input. |
| `LLM_SECURITY_AUDIT_LOG` | `audit.log_path` | unset — set a path to write the SIEM-ready JSONL audit trail. |
| `LLM_SECURITY_DEPRECATION_QUIET` | *(none)* | Silenced the deprecation warning; removed with the warning. |
```json
{
"injection": { "mode": "block" },
"trifecta": { "mode": "warn", "escalation_window": 5 },
"audit": { "log_path": "/var/log/llm-security/audit.jsonl" }
}
```
---
## 2. Sandboxing
@ -73,14 +90,16 @@ shell; avoid root, which disables user-namespace confinement.
### 3.1 Start in warn mode
Every new integration of `llm-security` should begin with all modes set to
`warn`. This yields advisories without breaking workflow, and lets the team
calibrate false-positive rates against their actual repositories.
`warn``injection.mode` and `trifecta.mode` in `.llm-security/policy.json`,
`LLM_SECURITY_PRECOMPACT_MODE` in the environment. This yields advisories
without breaking workflow, and lets the team calibrate false-positive rates
against their actual repositories.
### 3.2 Promote to block after baselining
After a baseline period (typically 1-2 weeks), flip each mode to `block` in this
order: `LLM_SECURITY_INJECTION_MODE`, `LLM_SECURITY_TRIFECTA_MODE`,
`LLM_SECURITY_PRECOMPACT_MODE`. The injection hook is first because false
order: `injection.mode`, `trifecta.mode` (both in `.llm-security/policy.json`),
then `LLM_SECURITY_PRECOMPACT_MODE`. The injection hook is first because false
positives there are the most visible; blocking comes last because the others
build confidence.
@ -183,7 +202,9 @@ tier:
Verdict cutoffs (`BLOCK ≥65`, `WARNING ≥15`) are locked to the `riskBand()`
boundaries so you can't get a "BLOCK / Medium band" contradiction. The legacy
formula is kept as `riskScoreV1()` for reference only.
v1 formula was kept as `riskScoreV1()` for reference through the v7 line and
was removed in v8.0.0; see `CHANGELOG.md` for the v1 weights if you need them
to re-derive an old score.
**CI impact:** Pipelines with `--fail-on high` keep working (the severity
gate is unaffected). Pipelines with score-based thresholds need recalibration

View file

@ -58,7 +58,7 @@ preview after step 3.
- **`hooks/scripts/post-session-guard.mjs`** — the only hook invoked.
Configurable via `policy.json` `trifecta.mode` (`block` / `warn` /
`off`; default `warn`) or env var `LLM_SECURITY_TRIFECTA_MODE`.
`off`; default `warn`) in `.llm-security/policy.json`.
This example uses `mode: warn` (default). In `block` mode the third
call's advisory becomes a hard block (exit 2) and the agent action is
@ -92,7 +92,7 @@ in a `finally` block before exiting. **Your real session state under
Jensen-Shannon divergence, or volume-threshold advisories.
Those have their own unit tests under `tests/lib/post-session-guard.*`.
- This is deterministic detection. It does not exercise the
`block`-mode exit-2 path — flip `LLM_SECURITY_TRIFECTA_MODE=block`
`block`-mode exit-2 path — set `"trifecta": {"mode": "block"}`
and re-run if you want to see the script fail at step 3.
## See also

View file

@ -20,7 +20,7 @@ The `systemMessage` payload from step 3 must contain:
- The literal phrase `Rule of Two violation`
- A list of evidence items under `Untrusted input:`, `Data access:`,
`Exfil sink:` headings
- A reference to `Set LLM_SECURITY_TRIFECTA_MODE=` for configuration
- A reference to the `trifecta.mode` policy key for configuration
- An OWASP tag mentioning `ASI01` or `ASI02`
Optional (depending on detail string and `policy.json` config):
@ -32,7 +32,7 @@ Optional (depending on detail string and `policy.json` config):
## Audit-trail side effect
When `LLM_SECURITY_AUDIT_LOG` (or `policy.json` `audit.log_path`) is
When the `policy.json` key `audit.log_path` is
set, step 3 writes a JSONL event:
```json
@ -48,7 +48,7 @@ set, step 3 writes a JSONL event:
The walkthrough does not configure the audit log — `writeAuditEvent`
no-ops when no path is set. To observe the audit-trail behaviour,
re-run with `LLM_SECURITY_AUDIT_LOG=/tmp/trifecta-audit.jsonl`.
re-run with `"audit": {"log_path": "/tmp/trifecta-audit.jsonl"}` in `.llm-security/policy.json`.
## State file

View file

@ -90,13 +90,12 @@ Expected: `5 pass, 0 fail`.
scope-hopping case the advisory is emitted before any network call.
For the clean case it may attempt `npm view` — that runs against
the public registry but is non-fatal if offline.
- **Stage B (dep-auditor)**: runs offline by default. If the env
var `LLM_SECURITY_OFFLINE=1` is unset, it may shell out to
`npm audit --json --offline=false` for CVE enrichment, but the
fixture has no real npm install, so audit returns nothing.
- **Stage B (dep-auditor)**: runs offline by default. It may shell
out to `npm audit --json --offline=false` for CVE enrichment, but
the fixture has no real npm install, so audit returns nothing.
If you need a fully air-gapped run, set `LLM_SECURITY_OFFLINE=1`
in the parent environment.
There is no environment kill-switch for these calls. If you need a
fully air-gapped run, block network egress for the process.
## OWASP / framework mapping

View file

@ -32,7 +32,7 @@ Attacker injects instructions via external content (files, web pages, tool outpu
| Prompt injection input scanning | Automated | `pre-prompt-inject-scan.mjs` detects CRITICAL/HIGH/MEDIUM injection patterns in user prompts | Hook file exists; MEDIUM advisory enabled |
| Unicode Tag steganography detection | Automated | `string-utils.mjs` decodes U+E0000-E007F tags; `injection-patterns.mjs` escalates to CRITICAL/HIGH | `decodeUnicodeTags()` in normalization pipeline |
| Bash evasion normalization | Automated | `bash-normalize.mjs` strips parameter expansion before pattern matching | `normalizeBashExpansion()` called by both bash hooks |
| Rule of Two detection (block-mode opt-in) | Automated | `post-session-guard.mjs` detects trifecta (untrusted input + sensitive data + exfil); blocks only when `LLM_SECURITY_TRIFECTA_MODE=block` AND high-confidence trifecta is observed; default `warn` | `LLM_SECURITY_TRIFECTA_MODE` env var respected; block mode opt-in |
| Rule of Two detection (block-mode opt-in) | Automated | `post-session-guard.mjs` detects trifecta (untrusted input + sensitive data + exfil); blocks only when the policy key `trifecta.mode` is `block` AND high-confidence trifecta is observed; default `warn` | `trifecta.mode` in `.llm-security/policy.json` respected; block mode opt-in |
| Long-horizon monitoring | Automated | `post-session-guard.mjs` 100-call window + behavioral drift detection | Long-horizon window active alongside 20-call window |
| HITL trap detection | Automated | `injection-patterns.mjs` HIGH patterns for approval urgency, summary suppression, scope minimization | HITL patterns present in HIGH_PATTERNS array |
| Hybrid attack detection | Automated | `injection-patterns.mjs` HYBRID_PATTERNS for P2SQL, recursive injection, XSS | Hybrid patterns checked in tool output scanning |

View file

@ -65,7 +65,7 @@ Research summary for the llm-security plugin. Documents what the field has learn
- Sensitive path patterns need expansion as new sensitive files emerge
**Plugin controls:**
- `post-session-guard.mjs`: `LLM_SECURITY_TRIFECTA_MODE=block|warn|off`
- `post-session-guard.mjs`: policy key `trifecta.mode` = `block|warn|off`
- Block mode: exit 2 for MCP-concentrated trifecta or sensitive path + exfil
- Default warn mode preserves backward compatibility
- **Gap:** Rule of Two is approximate — false positives possible for legitimate multi-tool workflows

View file

@ -9,9 +9,6 @@ export const SEVERITY = Object.freeze({
INFO: 'info',
});
// Legacy weights — used only by riskScoreV1() for backwards-compat reference.
const SEVERITY_WEIGHTS_V1 = { critical: 25, high: 10, medium: 4, low: 1, info: 0 };
/**
* Calculate aggregate risk score from severity counts (v2 model v7.0.0+).
*
@ -56,24 +53,6 @@ export function riskScore(counts) {
return Math.round(Math.min(100, base));
}
/**
* Legacy v1 risk score formula kept for diff/comparison only.
* Not exported in production paths; reference for CI re-calibration.
*
* @deprecated Since v7.0.0. Use riskScore() instead. Kept for diff/comparison only not used in production paths.
* @param {{ critical: number, high: number, medium: number, low: number, info: number }} counts
* @returns {number} 0-100 capped score (sum-and-cap model)
*/
export function riskScoreV1(counts) {
const raw =
(counts.critical || 0) * SEVERITY_WEIGHTS_V1.critical +
(counts.high || 0) * SEVERITY_WEIGHTS_V1.high +
(counts.medium || 0) * SEVERITY_WEIGHTS_V1.medium +
(counts.low || 0) * SEVERITY_WEIGHTS_V1.low +
(counts.info || 0) * SEVERITY_WEIGHTS_V1.info;
return Math.min(raw, 100);
}
/**
* Derive verdict from severity counts and risk score (v7.0.0 thresholds).
* Aligned to v2 riskBand cutoffs so verdict and band are co-monotonic:

View file

@ -1122,9 +1122,22 @@ async function checkRuleOfTwo(projectRoot, hooksJson) {
if (hookActive) {
const scriptPath = join(projectRoot, 'hooks', 'scripts', 'post-session-guard.mjs');
const content = await readText(scriptPath);
if (content && /TRIFECTA_MODE/i.test(content)) {
// What this category measures is whether enforcement is *configurable*, not
// what the constant holding it is called. v8.0.0 moved the mode to the
// policy.json key `trifecta.mode`; a pre-v8 vendored guard still resolving
// LLM_SECURITY_TRIFECTA_MODE is equally configurable and still passes.
const readsPolicyMode =
content && /getPolicyValue\w*\(\s*['"]trifecta['"]\s*,\s*['"]mode['"]/.test(content);
const readsLegacyEnv = content && /LLM_SECURITY_TRIFECTA_MODE/.test(content);
if (readsPolicyMode) {
hasTrifectaMode = true;
evidence.push('TRIFECTA_MODE configurable: yes');
evidence.push('Enforcement mode configurable: policy.json trifecta.mode');
} else if (readsLegacyEnv) {
hasTrifectaMode = true;
evidence.push(
'Enforcement mode configurable: LLM_SECURITY_TRIFECTA_MODE (pre-v8.0.0 env-var; ' +
'migrate to the policy.json key trifecta.mode)'
);
}
}
@ -1146,10 +1159,10 @@ async function checkRuleOfTwo(projectRoot, hooksJson) {
scanner: 'PST',
severity: SEVERITY.MEDIUM,
title: 'Session guard lacks configurable trifecta mode',
description: 'post-session-guard does not support LLM_SECURITY_TRIFECTA_MODE (block/warn/off).',
description: 'post-session-guard resolves no enforcement mode (block/warn/off) from the policy.json key trifecta.mode.',
file: 'hooks/scripts/post-session-guard.mjs',
owasp: 'ASI02',
recommendation: 'Upgrade to v5.0 session guard with configurable enforcement mode.',
recommendation: 'Upgrade to a session guard that reads trifecta.mode from .llm-security/policy.json.',
}));
return { status: STATUS.PARTIAL, findings, evidence };
}

View file

@ -1,10 +1,12 @@
#!/usr/bin/env node
// post-session-guard.mjs — Runtime trifecta detection (Rule of Two)
// v5.0: Configurable TRIFECTA_MODE (block|warn|off), long-horizon 100-call window,
// behavioral drift via Jensen-Shannon divergence
// v5.0: configurable enforcement mode (block|warn|off), long-horizon 100-call
// window, behavioral drift via Jensen-Shannon divergence
// v8.0.0: mode comes from the policy.json key trifecta.mode
import { readFileSync, appendFileSync } from 'node:fs';
import { getPolicyValue } from '../../scanners/lib/policy-loader.mjs';
const TRIFECTA_MODE = (process.env.LLM_SECURITY_TRIFECTA_MODE || 'warn').toLowerCase();
const TRIFECTA_MODE = String(getPolicyValue('trifecta', 'mode', 'warn')).toLowerCase();
const SLIDING_WINDOW = 20;
const LONG_HORIZON_WINDOW = 100;

View file

@ -242,3 +242,26 @@ describe('B11 — the env-var deprecation mechanism is gone from source', () =>
assert.deepEqual(offenders, [], `removed env-vars are still read:\n${offenders.join('\n')}`);
});
});
// ---------------------------------------------------------------------------
// riskScoreV1 — the other v7.3.0 deprecation shipping in the same major
// ---------------------------------------------------------------------------
describe('B11 — riskScoreV1 is removed', () => {
it('severity.mjs no longer exports riskScoreV1', async () => {
const mod = await import('../../scanners/lib/severity.mjs');
assert.equal(
mod.riskScoreV1,
undefined,
'the v1 sum-and-cap formula was @deprecated in v7.3.0 with zero consumers'
);
});
it('riskScore (v2) is untouched and still severity-dominated', async () => {
const { riskScore } = await import('../../scanners/lib/severity.mjs');
const oneCritical = riskScore({ critical: 1, high: 0, medium: 0, low: 0, info: 0 });
const manyLow = riskScore({ critical: 0, high: 0, medium: 0, low: 50, info: 0 });
assert.ok(oneCritical >= 70 && oneCritical <= 95, `one critical -> ${oneCritical}`);
assert.ok(manyLow < oneCritical, 'v1 collapsed this case to 100; v2 must not');
});
});

View file

@ -0,0 +1,129 @@
// posture-trifecta-mode.test.mjs — B11: posture category 12 (Rule of Two)
// recognises a policy-era session guard.
//
// Pre-v8 the check was `/TRIFECTA_MODE/i` over the hook source. That matched
// the *identifier* `TRIFECTA_MODE`, not the configuration mechanism, so a hook
// that resolves the mode from `.llm-security/policy.json` without naming a
// constant that way scored PARTIAL despite being strictly more configurable.
// With LLM_SECURITY_TRIFECTA_MODE removed in v8.0.0, that regex would push
// every correctly-migrated project off PASS.
import { describe, it, beforeEach, afterEach } from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, cpSync } from 'node:fs';
import { join, resolve } from 'node:path';
import { tmpdir } from 'node:os';
import { fileURLToPath } from 'node:url';
import { resetCounter } from '../../scanners/lib/output.mjs';
import { scan } from '../../scanners/posture-scanner.mjs';
const __dirname = fileURLToPath(new URL('.', import.meta.url));
const GRADE_A_FIXTURE = resolve(__dirname, '../fixtures/posture-scan/grade-a-project');
const GUARD_REL = join('hooks', 'scripts', 'post-session-guard.mjs');
// A guard that is configurable, but names nothing `TRIFECTA_MODE`.
const POLICY_ERA_GUARD = `#!/usr/bin/env node
// post-session-guard.mjs — Runtime trifecta detection (Rule of Two)
// v8.0.0: enforcement mode comes from .llm-security/policy.json.
import { readFileSync } from 'node:fs';
import { getPolicyValue } from '../../scanners/lib/policy-loader.mjs';
const mode = String(getPolicyValue('trifecta', 'mode', 'warn')).toLowerCase();
const input = JSON.parse(readFileSync('/dev/stdin', 'utf-8'));
if (mode === 'off') {
process.stdout.write(JSON.stringify({ decision: 'allow' }));
process.exit(0);
}
const LONG_HORIZON_WINDOW = 100;
process.stdout.write(JSON.stringify({ decision: 'allow', window: LONG_HORIZON_WINDOW }));
`;
// A guard with no configurable mode at all — hardcoded enforcement.
const UNCONFIGURABLE_GUARD = `#!/usr/bin/env node
// post-session-guard.mjs — Runtime trifecta detection, no configuration.
import { readFileSync } from 'node:fs';
const input = JSON.parse(readFileSync('/dev/stdin', 'utf-8'));
const LONG_HORIZON_WINDOW = 100;
process.stdout.write(JSON.stringify({ decision: 'allow', window: LONG_HORIZON_WINDOW }));
`;
let root = null;
/** Copy of grade-a-project whose session guard is replaced with `source`. */
function projectWithGuard(source) {
root = mkdtempSync(join(tmpdir(), 'llmsec-posture-trifecta-'));
cpSync(GRADE_A_FIXTURE, root, { recursive: true });
mkdirSync(join(root, 'hooks', 'scripts'), { recursive: true });
writeFileSync(join(root, GUARD_REL), source);
return root;
}
function ruleOfTwo(result) {
return result.categories.find((c) => c.id === 12);
}
/** Category findings live on the top-level `findings` array, not on the category. */
function ruleOfTwoFindings(result) {
return result.findings.filter((f) => /trifecta mode|Rule of Two/i.test(f.title));
}
describe('posture category 12 — Rule of Two mode detection (B11)', () => {
afterEach(() => {
if (root) rmSync(root, { recursive: true, force: true });
root = null;
});
beforeEach(() => {
resetCounter();
});
it('PASSes a policy-era guard that reads trifecta.mode from policy.json', async () => {
const result = await scan(projectWithGuard(POLICY_ERA_GUARD));
const cat = ruleOfTwo(result);
assert.equal(
cat.status,
'PASS',
'a guard configured through policy.json is configurable; the identifier name is not the mechanism'
);
assert.ok(
cat.evidence.some((e) => /trifecta\.mode|policy\.json/i.test(e)),
`expected evidence naming the policy key, got: ${JSON.stringify(cat.evidence)}`
);
});
it('PASSes a legacy guard still using the pre-v8 env-var', async () => {
// Third-party projects vendor their own hook copy and may not have migrated.
// Their enforcement mode is still configurable, so the category still holds.
const legacy = POLICY_ERA_GUARD.replace(
"String(getPolicyValue('trifecta', 'mode', 'warn'))",
"String(process.env.LLM_SECURITY_TRIFECTA_MODE || 'warn')"
);
const result = await scan(projectWithGuard(legacy));
assert.equal(ruleOfTwo(result).status, 'PASS');
});
it('does not PASS a guard with no configurable mode', async () => {
const result = await scan(projectWithGuard(UNCONFIGURABLE_GUARD));
const cat = ruleOfTwo(result);
assert.equal(cat.status, 'PARTIAL', 'hardcoded enforcement is not a configurable mode');
assert.ok(
ruleOfTwoFindings(result).some((f) => /configurable trifecta mode/i.test(f.title)),
'expected the PARTIAL finding to name the missing configurability'
);
});
it('the PARTIAL finding recommends the policy key, not a removed env-var', async () => {
const result = await scan(projectWithGuard(UNCONFIGURABLE_GUARD));
const f = ruleOfTwoFindings(result).find((x) => /configurable trifecta mode/i.test(x.title));
assert.ok(f, 'expected the PARTIAL finding');
const text = `${f.description} ${f.recommendation}`;
assert.doesNotMatch(
text,
/LLM_SECURITY_TRIFECTA_MODE/,
'must not tell the user to set an env-var that v8 ignores'
);
assert.match(text, /trifecta\.mode/, 'should name the policy key');
});
});