open/llm-security
1
0
Fork 0
Code Issues Releases 1 Activity
llm-security/tests/scanners
History
Kjell Tore Guttormsen eceb71bbb3 fix(llm-security): SIG self-flagged the vendored commons it detects from 
Scanning this repository with the SIG scanner produced 7 findings, 4 of them
on our own detection data: scanners/commons/CHANGELOG.md and
scanners/commons/signatures/malware-signatures.json. The ruleset that describes
xmrig and webshells is, byte for byte, a document containing those strings, so
the engine matched it as malware. EXCLUDED_PATH_RE already carried
knowledge/, tests/, docs/ and node_modules/ for exactly this reason; the
vendored commons arrived in v8 Phase 5 (bbada84) without being added.

One alternation branch closes it. Tests first: two cases added to
describe('signature-scanner: path exclusions'), both verified red against the
real scan() entry point before the regex changed.

Stated plainly, because it is a real cost and not a technicality: the branch is
`scanners\/commons` behind the existing `(^|\/)` prefix, so it matches that
two-segment path ANYWHERE in a target's relative path, not only at its root. A
webshell planted at vendor/scanners/commons/shell.php in a hostile repository is
therefore invisible to SIG. The second new test asserts that blind spot
deliberately, so it can never be discovered by accident. It is accepted because
anchoring at ^scanners/commons/ would miss the same payload one directory
deeper while re-opening the self-flag whenever the plugin is scanned from a
parent directory. TRG, AST, entropy and supply-chain still read these files;
only SIG identity-matching is blinded.

scanners/lib/supply-chain-data.mjs is NOT excluded. Its finding is a true
positive against real blocklist data.

Measured before: 7 findings. After: 3 (2 on STATE.md, 1 on
supply-chain-data.mjs). signature-scanner.test.mjs 23/23; custom-rules + e2e
54/54; golden-baseline 8/8 with suite-counts.json untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SBJVYzwch63Rpk1hii1cNM
2026-08-13 21:41:09 +02:00
..
ai-bom.test.mjs feat(scanner): add AI-BOM generator — CycloneDX 1.6 format for AI supply chain transparency 2026-04-10 13:29:30 +02:00
ast-taint-scanner.test.mjs fix(llm-security): AST-taint clears taint on reassignment (#29) + sink test coverage (#28) 2026-07-18 10:14:51 +02:00
attack-simulator.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
auto-cleaner-rce.test.mjs fix(llm-security): CRITICAL — command injection in auto-cleaner (v7.8.1) 2026-07-18 08:48:09 +02:00
auto-cleaner-traversal.test.mjs fix(llm-security): F-2/F-3 — contain path traversal + kill npm-view shell injection 2026-06-20 11:09:13 +02:00
auto-cleaner.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
bash-normalize-t5-t6.test.mjs fix(llm-security): normalization/discovery evasion + SIG embedded-base64 & custom rules (#21,#23,#30,#36,#42,#52,#55) 2026-07-18 10:35:56 +02:00
bash-normalize-t7-t9.test.mjs feat(bash-normalize): T7 — process substitution collapse (E8) 2026-04-30 15:14:04 +02:00
benchmark.test.mjs feat(scanner): add --benchmark mode to attack-simulator with structured reporting 2026-04-10 13:02:58 +02:00
ci-integration.test.mjs feat(ci): add CI/CD integration — --fail-on, --compact, pipeline templates 2026-04-10 14:59:05 +02:00
cli-wrapper.test.mjs feat(cli): add standalone CLI wrapper — npx llm-security scan without Claude Code 2026-04-10 13:58:25 +02:00
compliance-mapping.test.mjs feat(knowledge): add compliance-mapping document — EU AI Act, NIST AI RMF, ISO 42001 2026-04-10 12:29:14 +02:00
content-extractor-strip.test.mjs fix(llm-security): v8 Phase 1 — Berry lockfile, nested-v1 recursion, per-occurrence strip attribution 2026-08-02 21:10:47 +02:00
dashboard.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
dep-token-overlap.test.mjs fix(dep): B7 — token-overlap typosquat heuristic alongside Levenshtein 2026-04-29 14:10:53 +02:00
dep.test.mjs fix(llm-security): supply-chain gate bypasses — npm/yarn blocklist + pip-audit (#14-#19,#48) 2026-07-18 10:14:51 +02:00
entropy-context.test.mjs fix(entropy): E18 — rule 18 markdown-image CDN-aware + secret pre-check 2026-04-29 15:18:37 +02:00
entropy-path-suppression.test.mjs fix(llm-security): HIGH — entropy suppression keyed off the absolute path 2026-07-18 09:11:04 +02:00
entropy.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
git-injection.test.mjs fix(llm-security): F-1 — eliminate shell-injection RCE in git-forensics scanner 2026-06-20 10:18:41 +02:00
git-reflog-reset.test.mjs fix(llm-security): misc scanner correctness — reflog FP, diff exact-pass, hex dedupe, SARIF version (#20,#22,#50,#54,#56) 2026-07-18 10:36:51 +02:00
git.test.mjs test(llm-security): loosen git-forensics finding count thresholds 2026-04-18 11:00:20 +02:00
ide-extension-data.test.mjs fix(llm-security): misc scanner correctness — reflog FP, diff exact-pass, hex dedupe, SARIF version (#20,#22,#50,#54,#56) 2026-07-18 10:36:51 +02:00
ide-extension-discovery.test.mjs feat(llm-security): implement JetBrains discovery + Android Studio base dir 2026-04-18 10:16:28 +02:00
ide-extension-null-manifest.test.mjs chore(llm-security): v7.8.2 — security patch release 2026-07-18 09:33:40 +02:00
ide-extension-parser-entities.test.mjs fix(llm-security): out-of-range char-ref silently emptied a plugin.xml field 2026-07-18 09:21:56 +02:00
ide-extension-scanner.test.mjs test(llm-security): add end-to-end JetBrains scan integration tests 2026-04-18 10:51:48 +02:00
ide-extension-url.test.mjs feat(llm-security): OS sandbox for /security ide-scan <url> (v6.5.0) 2026-04-17 17:28:57 +02:00
jetbrains-fetch.test.mjs feat(llm-security): URL-fetch support for JetBrains Marketplace (v6.6.0) 2026-04-18 10:46:13 +02:00
jetbrains-parser.test.mjs feat(llm-security): implement parseIntelliJPlugin with nested-jar extraction 2026-04-18 10:15:12 +02:00
knowledge-atlas.test.mjs feat(knowledge): add MITRE ATLAS IDs to OWASP files + Norwegian regulatory context 2026-04-10 12:49:10 +02:00
mcp-baseline-reset.test.mjs feat(commands): E14 part 3 — /security mcp-baseline-reset slash command 2026-04-30 16:49:01 +02:00
mcp-live-inspect-stdout-cap.test.mjs fix(llm-security): scanner robustness — ReDoS, MCP-stdout DoS, redirect loop, atomic writes (#24,#53,#31,#25,#51) 2026-07-18 10:15:11 +02:00
memory-poisoning-hex-dedupe.test.mjs fix(llm-security): misc scanner correctness — reflog FP, diff exact-pass, hex dedupe, SARIF version (#20,#22,#50,#54,#56) 2026-07-18 10:36:51 +02:00
memory-poisoning.test.mjs fix(memory-poisoning): E15 — add .claude/agents/*.md to target glob 2026-04-29 14:13:01 +02:00
network.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
permission.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
posture-trifecta-mode.test.mjs feat(llm-security)!: v8 Phase 3 complete - riskScoreV1, posture heuristic, docs 2026-08-09 10:25:03 +02:00
posture.test.mjs feat(posture): add EU AI Act, NIST AI RMF, ISO 42001 compliance categories (14-16) 2026-04-10 13:17:25 +02:00
reference-config.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
sarif-version.test.mjs fix(llm-security): misc scanner correctness — reflog FP, diff exact-pass, hex dedupe, SARIF version (#20,#22,#50,#54,#56) 2026-07-18 10:36:51 +02:00
sarif.test.mjs feat(scanner): add SARIF 2.1.0 output format to scan-orchestrator (--format sarif) 2026-04-10 13:22:59 +02:00
signature-scanner-custom-rules.test.mjs fix(llm-security): normalization/discovery evasion + SIG embedded-base64 & custom rules (#21,#23,#30,#36,#42,#52,#55) 2026-07-18 10:35:56 +02:00
signature-scanner.test.mjs fix(llm-security): SIG self-flagged the vendored commons it detects from 2026-08-13 21:41:09 +02:00
skill-scanner-narrative.test.mjs test(llm-security): narrative-coherence contract test (v7.1.1) 2026-04-29 12:50:27 +02:00
supply-chain-recheck.test.mjs fix(llm-security): v8 Phase 1 — Berry lockfile, nested-v1 recursion, per-occurrence strip attribution 2026-08-02 21:10:47 +02:00
taint-destructuring.test.mjs fix(taint-tracer): B6 — recognize destructuring + spread + rest patterns 2026-04-29 14:05:34 +02:00
taint-tracer.test.mjs feat(llm-security): add .kt .groovy .scala to taint-tracer CODE_EXTENSIONS 2026-04-18 10:04:32 +02:00
taint.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
toxic-flow-keyword-boundary.test.mjs fix(llm-security): scanner false-positives/negatives — trigger, toxic-flow, policy-loader (#38-#41,#57,#26) 2026-07-18 10:15:10 +02:00
trigger-scanner.test.mjs fix(llm-security): scanner false-positives/negatives — trigger, toxic-flow, policy-loader (#38-#41,#57,#26) 2026-07-18 10:15:10 +02:00
unicode-bom.test.mjs fix(llm-security): normalization/discovery evasion + SIG embedded-base64 & custom rules (#21,#23,#30,#36,#42,#52,#55) 2026-07-18 10:35:56 +02:00
unicode.test.mjs feat(ultraplan-local): v1.6.0 — /ultraresearch-local deep research command 2026-04-08 08:58:35 +02:00
vsix-fetch.test.mjs fix(llm-security): scanner robustness — ReDoS, MCP-stdout DoS, redirect loop, atomic writes (#24,#53,#31,#25,#51) 2026-07-18 10:15:11 +02:00
vsix-sandbox.test.mjs refactor(llm-security): parameterize buildSandboxedWorker with workerPath 2026-04-18 10:37:10 +02:00
workflow-scanner.test.mjs fix(llm-security): YAML/workflow parser divergence — block scalars + bare if: (#32,#33,#43) 2026-07-18 10:35:56 +02:00
zip-extract.test.mjs feat(llm-security): /security ide-scan <url> — Marketplace/OpenVSX/direct VSIX (v6.4.0) 2026-04-17 17:16:26 +02:00