llm-security/tests/scanners
Kjell Tore Guttormsen fdec4b36ad feat(llm-security)!: v8 Phase 3 complete - riskScoreV1, posture heuristic, docs
Closes Phase 3 (B11) of the v8.0.0 plan. Three parts, all with the failing
test written first.

riskScoreV1 removed. scanners/lib/severity.mjs drops riskScoreV1() and its
SEVERITY_WEIGHTS_V1 table - @deprecated since v7.0.0, kept for diff/comparison,
zero callers in code or tests (re-verified, not taken from the plan). The v1
weights are recorded in CHANGELOG so an old score stays re-derivable. riskScore
(v2) is untouched; a test pins that one critical still lands in the 70-95 tier
and that 50 lows score below it, which is exactly the case v1 collapsed to 100.

Posture category 12 no longer keys off an identifier name. The check was
/TRIFECTA_MODE/i over the session-guard source, which measured what a constant
was CALLED rather than whether enforcement was configurable. With the env-var
gone, that regex would have dropped every correctly-migrated project from PASS
to PARTIAL - the gate punishing the migration it exists to encourage. It now
matches getPolicyValue('trifecta', 'mode', ...) and still accepts a pre-v8
vendored guard reading the old env-var, because a third-party project carries
its own hook copy and is equally configurable either way; the evidence line
says which of the two was found. The PARTIAL finding recommended setting an
env-var that v8 ignores; it now names the policy key. The grade-a fixture hook
moves to the policy-era form.

Two never-implemented env-vars deleted from the docs. LLM_SECURITY_SCR_OFFLINE
(ci-cd-guide) and LLM_SECURITY_OFFLINE (supply-chain-attack example) were
documented as OSV.dev / npm-audit kill-switches. No code has ever read either -
verified by grep across scanners, hooks and scripts, which finds them only in
markdown. A promised kill-switch that does nothing is worse than a documented
absence: it is trusted precisely when the run is meant to be air-gapped. The
docs now say there is none and that egress must be blocked at the network
layer. The LLM_SECURITY_AUDIT_* wildcard is narrowed to the one real key.

Docs. Migration section in README + CHANGELOG with the env-var -> policy-key
table, the detection commands (env + shell rc + .envrc + workflows), and the
explicit warning that a removed variable is now INERT rather than an error -
which is the failure mode that loses a project its configuration silently. The
hardening-guide env table splits into surviving vars and a removed-vars
migration table; its "promote to block" runbook named two variables that no
longer exist. Also swept: CLAUDE.md hook table, scanner-reference, ci-cd-guide,
both lethal-trifecta example docs, mitigation-matrix, injection-research.

Test counts in README/CLAUDE.md synced 2034 -> 2045.

Suite 2045 tests, 0 fail (2039 + 4 posture-trifecta + 2 riskScoreV1). The two
known parallel-load flakes did not recur this run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
2026-08-09 10:25:03 +02:00
..
ai-bom.test.mjs feat(scanner): add AI-BOM generator — CycloneDX 1.6 format for AI supply chain transparency 2026-04-10 13:29:30 +02:00
ast-taint-scanner.test.mjs fix(llm-security): AST-taint clears taint on reassignment (#29) + sink test coverage (#28) 2026-07-18 10:14:51 +02:00
attack-simulator.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
auto-cleaner-rce.test.mjs fix(llm-security): CRITICAL — command injection in auto-cleaner (v7.8.1) 2026-07-18 08:48:09 +02:00
auto-cleaner-traversal.test.mjs fix(llm-security): F-2/F-3 — contain path traversal + kill npm-view shell injection 2026-06-20 11:09:13 +02:00
auto-cleaner.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
bash-normalize-t5-t6.test.mjs fix(llm-security): normalization/discovery evasion + SIG embedded-base64 & custom rules (#21,#23,#30,#36,#42,#52,#55) 2026-07-18 10:35:56 +02:00
bash-normalize-t7-t9.test.mjs feat(bash-normalize): T7 — process substitution collapse (E8) 2026-04-30 15:14:04 +02:00
benchmark.test.mjs feat(scanner): add --benchmark mode to attack-simulator with structured reporting 2026-04-10 13:02:58 +02:00
ci-integration.test.mjs feat(ci): add CI/CD integration — --fail-on, --compact, pipeline templates 2026-04-10 14:59:05 +02:00
cli-wrapper.test.mjs feat(cli): add standalone CLI wrapper — npx llm-security scan without Claude Code 2026-04-10 13:58:25 +02:00
compliance-mapping.test.mjs feat(knowledge): add compliance-mapping document — EU AI Act, NIST AI RMF, ISO 42001 2026-04-10 12:29:14 +02:00
content-extractor-strip.test.mjs fix(llm-security): v8 Phase 1 — Berry lockfile, nested-v1 recursion, per-occurrence strip attribution 2026-08-02 21:10:47 +02:00
dashboard.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
dep-token-overlap.test.mjs fix(dep): B7 — token-overlap typosquat heuristic alongside Levenshtein 2026-04-29 14:10:53 +02:00
dep.test.mjs fix(llm-security): supply-chain gate bypasses — npm/yarn blocklist + pip-audit (#14-#19,#48) 2026-07-18 10:14:51 +02:00
entropy-context.test.mjs fix(entropy): E18 — rule 18 markdown-image CDN-aware + secret pre-check 2026-04-29 15:18:37 +02:00
entropy-path-suppression.test.mjs fix(llm-security): HIGH — entropy suppression keyed off the absolute path 2026-07-18 09:11:04 +02:00
entropy.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
git-injection.test.mjs fix(llm-security): F-1 — eliminate shell-injection RCE in git-forensics scanner 2026-06-20 10:18:41 +02:00
git-reflog-reset.test.mjs fix(llm-security): misc scanner correctness — reflog FP, diff exact-pass, hex dedupe, SARIF version (#20,#22,#50,#54,#56) 2026-07-18 10:36:51 +02:00
git.test.mjs test(llm-security): loosen git-forensics finding count thresholds 2026-04-18 11:00:20 +02:00
ide-extension-data.test.mjs fix(llm-security): misc scanner correctness — reflog FP, diff exact-pass, hex dedupe, SARIF version (#20,#22,#50,#54,#56) 2026-07-18 10:36:51 +02:00
ide-extension-discovery.test.mjs feat(llm-security): implement JetBrains discovery + Android Studio base dir 2026-04-18 10:16:28 +02:00
ide-extension-null-manifest.test.mjs chore(llm-security): v7.8.2 — security patch release 2026-07-18 09:33:40 +02:00
ide-extension-parser-entities.test.mjs fix(llm-security): out-of-range char-ref silently emptied a plugin.xml field 2026-07-18 09:21:56 +02:00
ide-extension-scanner.test.mjs test(llm-security): add end-to-end JetBrains scan integration tests 2026-04-18 10:51:48 +02:00
ide-extension-url.test.mjs feat(llm-security): OS sandbox for /security ide-scan <url> (v6.5.0) 2026-04-17 17:28:57 +02:00
jetbrains-fetch.test.mjs feat(llm-security): URL-fetch support for JetBrains Marketplace (v6.6.0) 2026-04-18 10:46:13 +02:00
jetbrains-parser.test.mjs feat(llm-security): implement parseIntelliJPlugin with nested-jar extraction 2026-04-18 10:15:12 +02:00
knowledge-atlas.test.mjs feat(knowledge): add MITRE ATLAS IDs to OWASP files + Norwegian regulatory context 2026-04-10 12:49:10 +02:00
mcp-baseline-reset.test.mjs feat(commands): E14 part 3 — /security mcp-baseline-reset slash command 2026-04-30 16:49:01 +02:00
mcp-live-inspect-stdout-cap.test.mjs fix(llm-security): scanner robustness — ReDoS, MCP-stdout DoS, redirect loop, atomic writes (#24,#53,#31,#25,#51) 2026-07-18 10:15:11 +02:00
memory-poisoning-hex-dedupe.test.mjs fix(llm-security): misc scanner correctness — reflog FP, diff exact-pass, hex dedupe, SARIF version (#20,#22,#50,#54,#56) 2026-07-18 10:36:51 +02:00
memory-poisoning.test.mjs fix(memory-poisoning): E15 — add .claude/agents/*.md to target glob 2026-04-29 14:13:01 +02:00
network.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
permission.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
posture-trifecta-mode.test.mjs feat(llm-security)!: v8 Phase 3 complete - riskScoreV1, posture heuristic, docs 2026-08-09 10:25:03 +02:00
posture.test.mjs feat(posture): add EU AI Act, NIST AI RMF, ISO 42001 compliance categories (14-16) 2026-04-10 13:17:25 +02:00
reference-config.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
sarif-version.test.mjs fix(llm-security): misc scanner correctness — reflog FP, diff exact-pass, hex dedupe, SARIF version (#20,#22,#50,#54,#56) 2026-07-18 10:36:51 +02:00
sarif.test.mjs feat(scanner): add SARIF 2.1.0 output format to scan-orchestrator (--format sarif) 2026-04-10 13:22:59 +02:00
signature-scanner-custom-rules.test.mjs fix(llm-security): normalization/discovery evasion + SIG embedded-base64 & custom rules (#21,#23,#30,#36,#42,#52,#55) 2026-07-18 10:35:56 +02:00
signature-scanner.test.mjs test(llm-security): cover TRG/SIG/AST in e2e pipeline + SIG miner/hacktool/rot13 families (#58,#59) 2026-07-18 10:46:19 +02:00
skill-scanner-narrative.test.mjs test(llm-security): narrative-coherence contract test (v7.1.1) 2026-04-29 12:50:27 +02:00
supply-chain-recheck.test.mjs fix(llm-security): v8 Phase 1 — Berry lockfile, nested-v1 recursion, per-occurrence strip attribution 2026-08-02 21:10:47 +02:00
taint-destructuring.test.mjs fix(taint-tracer): B6 — recognize destructuring + spread + rest patterns 2026-04-29 14:05:34 +02:00
taint-tracer.test.mjs feat(llm-security): add .kt .groovy .scala to taint-tracer CODE_EXTENSIONS 2026-04-18 10:04:32 +02:00
taint.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
toxic-flow-keyword-boundary.test.mjs fix(llm-security): scanner false-positives/negatives — trigger, toxic-flow, policy-loader (#38-#41,#57,#26) 2026-07-18 10:15:10 +02:00
trigger-scanner.test.mjs fix(llm-security): scanner false-positives/negatives — trigger, toxic-flow, policy-loader (#38-#41,#57,#26) 2026-07-18 10:15:10 +02:00
unicode-bom.test.mjs fix(llm-security): normalization/discovery evasion + SIG embedded-base64 & custom rules (#21,#23,#30,#36,#42,#52,#55) 2026-07-18 10:35:56 +02:00
unicode.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
vsix-fetch.test.mjs fix(llm-security): scanner robustness — ReDoS, MCP-stdout DoS, redirect loop, atomic writes (#24,#53,#31,#25,#51) 2026-07-18 10:15:11 +02:00
vsix-sandbox.test.mjs refactor(llm-security): parameterize buildSandboxedWorker with workerPath 2026-04-18 10:37:10 +02:00
workflow-scanner.test.mjs fix(llm-security): YAML/workflow parser divergence — block scalars + bare if: (#32,#33,#43) 2026-07-18 10:35:56 +02:00
zip-extract.test.mjs feat(llm-security): /security ide-scan <url> — Marketplace/OpenVSX/direct VSIX (v6.4.0) 2026-04-17 17:16:26 +02:00