feat(llm-security)!: v7.0.0 commit 6 — tests, docs, version bump
Final commit in the trustworthy-scoring series. Bundles verdict cutoff
alignment, the last suite of tests, and all documentation touch-points
that quote version numbers or describe v7.0.0 behaviour.
Verdict/band co-monotonicity
- `scanners/lib/severity.mjs` — verdict cutoffs moved from 61/21 to 65/15
so `BLOCK >= 65`, `WARNING >= 15` locks onto the v2 riskBand() boundaries.
Prevents "BLOCK / Medium band" contradictions under the v2 formula.
Scanner hardening (bug fixes from v7.0.0 testing)
- `scanners/entropy-scanner.mjs` — `policy_source` now uses
`existsSync('.llm-security/policy.json')` instead of value-based check.
Old heuristic always reported 'policy.json' because DEFAULT_POLICY now
carries an `entropy.thresholds` section.
- `scanners/lib/file-discovery.mjs` — `.sass` and GPU shader extensions
(`.glsl, .frag, .vert, .shader, .wgsl`) added to TEXT_EXTENSIONS. Without
this, shader files were invisible to file-discovery, so they were never
counted as skipped by the entropy-scanner extension filter.
Tests
- `tests/scanners/entropy-context.test.mjs` (new, 24 tests) — A. File-ext
skip (4), B. Line-level rules 11-17 (8), C. Policy overrides (3).
Fixtures generate 80-char base64 payloads at runtime via
`crypto.randomBytes` to dodge the plugin's own pre-edit credential hook
on the test source.
- `tests/lib/severity.test.mjs` — rewritten with v2 scoring table (70
tests total, was 52).
- `tests/lib/output.test.mjs:243` — "1 critical = score 80" under v2
(was 25 under v1).
- Full suite: 1485/1485 green (was 1461).
Docs
- `CHANGELOG.md` — v7.0.0 entry with BREAKING CHANGES section.
- `README.md` (plugin + marketplace root) — version badge, history table,
plugin-card version string, test count.
- `CLAUDE.md` — header version, "v7.0.0 — Trustworthy scoring" summary
paragraph at the top.
- `docs/security-hardening-guide.md` — new section 6 "Calibration & false
positives" documenting v2 formula, context-aware entropy scanner,
typosquat allowlist, and §6.4 tuning workflow. Existing "Recommended
baseline" section renumbered to §7.
Version bump
- `6.6.0 -> 7.0.0` across package.json, .claude-plugin/plugin.json,
scanners/ide-extension-scanner.mjs VERSION const, README badge,
CLAUDE.md header, marketplace root README card.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen
parent
915aca69e4
commit
6f86de937a
14 changed files with 515 additions and 85 deletions
|
|
@ -26,7 +26,7 @@ Then open Claude Code and type `/plugin` to browse and install plugins from the
|
|||
|
||||
## Plugins
|
||||
|
||||
### [LLM Security](plugins/llm-security/) `v6.6.0`
|
||||
### [LLM Security](plugins/llm-security/) `v7.0.0`
|
||||
|
||||
Security scanning, auditing, and threat modeling for agentic AI projects.
|
||||
|
||||
|
|
@ -40,7 +40,7 @@ Built on OWASP LLM Top 10 (2025), OWASP Agentic AI Top 10, and the AI Agent Trap
|
|||
|
||||
Key commands: `/security posture`, `/security audit`, `/security scan`, `/security ide-scan`, `/security threat-model`, `/security plugin-audit`
|
||||
|
||||
6 specialized agents · 22 scanners · 9 hooks · 20 knowledge docs · 1461 tests
|
||||
6 specialized agents · 22 scanners · 9 hooks · 20 knowledge docs · 1485 tests
|
||||
|
||||
→ [Full documentation](plugins/llm-security/README.md)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
{
|
||||
"name": "llm-security",
|
||||
"description": "Security scanning, auditing, and threat modeling for Claude Code projects. Detects secrets, validates MCP servers, assesses security posture, and generates threat models aligned with OWASP LLM Top 10.",
|
||||
"version": "6.6.0"
|
||||
"version": "7.0.0"
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,6 +4,44 @@ All notable changes to the LLM Security Plugin are documented in this file.
|
|||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## [7.0.0] - 2026-04-19
|
||||
|
||||
### BREAKING CHANGES
|
||||
- **Risk-score formula rewritten** (`scanners/lib/severity.mjs`). The v1 sum-and-cap formula (`critical*25 + high*10 + medium*4 + low*1`, capped at 100) collapsed every non-trivial scan to 100/Extreme regardless of actual risk distribution. v2 is severity-dominated and log-scaled within tier:
|
||||
- Critical present → 70–95 (1=80, 2=86, 4=90, 10=95)
|
||||
- High only → 40–65 (1=48, 5=60, 17=65)
|
||||
- Medium only → 15–35 (1=20, 5=28, 50=33)
|
||||
- Low only → 1–11 (1=4, 10=11)
|
||||
- None → 0
|
||||
Verdict cutoffs realigned to new bands: `BLOCK` if critical ≥1 or score ≥65, `WARNING` if high ≥1 or score ≥15. Legacy v1 formula kept as `riskScoreV1()` for reference only. CI pipelines with `--fail-on` thresholds may need recalibration — see `docs/security-hardening-guide.md` §6.
|
||||
- **Verdict/band cutoffs aligned for co-monotonicity.** Old cutoffs (BLOCK ≥61, WARNING ≥21) could produce "BLOCK / Medium band" or "ALLOW / High band" contradictions. New cutoffs (65, 15) are locked to the v2 `riskBand()` boundaries.
|
||||
|
||||
### Added
|
||||
- **Context-aware entropy scanner** (`scanners/entropy-scanner.mjs`). Skip-lists and line-level rules drastically reduce false positives in shader/CSS/HTML/SQL-heavy codebases:
|
||||
- File-extension skip: `.glsl, .frag, .vert, .shader, .wgsl, .css, .scss, .sass, .less, .svg` + compound `.min.js, .min.css, .map`
|
||||
- Line-level rules 11–17 in `isFalsePositive()`: GLSL keywords (`uniform`, `vec3`, `texture2D`...), CSS-in-JS templates (`styled.`), inline `<svg>` markup, ffmpeg `filter_complex` syntax, browser `User-Agent` strings, SQL DDL on dedicated lines (`^\s*(SELECT|INSERT|UPDATE|DELETE|CREATE|...)`), `throw new Error(\`…\`)` templates
|
||||
- Scanner envelope gains `calibration` block: `files_skipped_by_extension`, `files_skipped_by_path`, effective `thresholds`, and `policy_source` (`'default' | 'policy.json'`)
|
||||
- **Policy-driven entropy configuration** — `.llm-security/policy.json` `entropy` section accepts:
|
||||
- `thresholds.{critical,high,medium}.{entropy,minLen}` — override defaults per project
|
||||
- `suppress_extensions: string[]` — additional file extensions to skip
|
||||
- `suppress_line_patterns: string[]` — user-defined regexes for line suppression
|
||||
- `suppress_paths: string[]` — substring match against `relPath` to skip entire paths (e.g., `"vendored/"`)
|
||||
- **DEP typosquat allowlist expansion** (`knowledge/typosquat-allowlist.json`). 22 npm + 5 PyPI entries for short-name modern tools that tripped Levenshtein detection on nearly every real codebase:
|
||||
- npm: `knip`, `oxlint`, `tsx`, `nx`, `rimraf`, `glob`, `tar`, `zod`, `ky`, `ow`, `esm`, `ip`, `qs`, `url`, `prettier`, `vitest`, `vite`, `rollup`, `swc`, `turbo`, `bun`, `deno`
|
||||
- PyPI: `uv`, `ruff`, `rich`, `typer`, `anyio`
|
||||
- **Synthesizer "Scan Calibration" section** (`agents/deep-scan-synthesizer-agent.md`). Heuristic: omit if <5% files skipped, flag prominently if >80% skipped by path (signals over-aggressive user policy). Agent instructed to NEVER override scanner verdict with narrative opinion.
|
||||
- **24 new unit tests** (`tests/scanners/entropy-context.test.mjs`): A. File-extension skip (4), B. Line-level rules 11–17 (8), C. Policy overrides (3); plus expanded `tests/lib/severity.test.mjs` with v2 scoring/band/verdict tables (70 tests total, was 52). **Total: 1485 tests (was 1461).**
|
||||
|
||||
### Changed
|
||||
- `tests/lib/output.test.mjs:243` — "1 critical = score 80" under v2 (was 25 under v1).
|
||||
- `scanners/lib/file-discovery.mjs` — `TEXT_EXTENSIONS` now includes `.sass` and GPU shader source extensions (`.glsl, .frag, .vert, .shader, .wgsl`) so these files are discovered and explicitly counted as skipped by the entropy scanner instead of invisibly filtered out.
|
||||
- Plugin version: `6.6.0 → 7.0.0` across `package.json`, `.claude-plugin/plugin.json`, `scanners/ide-extension-scanner.mjs` (`VERSION`), README badge, CLAUDE.md header, marketplace root README.
|
||||
|
||||
### Why
|
||||
- **Real-world scan on `hyperframes.com` produced `BLOCK / Extreme / 100` with ~70% noise** (shader strings, CSS gradients, bundled JS, Levenshtein false positives). A scanner that cries "extreme" on every project destroys its own credibility — users learn to ignore findings, so genuine threats slip past.
|
||||
- **Trustworthiness comes from calibration, not from detecting everything.** v7.0.0 accepts that some detection heuristics are noisy in context (entropy on shaders, typosquat on 2–3 letter tool names) and gives users both built-in suppression and policy-driven override controls.
|
||||
- **Verdict/score/band co-monotonicity fixed.** A user can now correctly reason: "HIGH band → WARNING verdict" without reading the source. The v1 cutoffs allowed a mid-High score (42) to produce ALLOW and a low-Medium score (22) to produce WARNING.
|
||||
|
||||
## [6.6.0] - 2026-04-18
|
||||
|
||||
### Added
|
||||
|
|
|
|||
|
|
@ -1,6 +1,14 @@
|
|||
# LLM Security Plugin (v6.6.0)
|
||||
# LLM Security Plugin (v7.0.0)
|
||||
|
||||
Security scanning, auditing, and threat modeling for Claude Code projects. 5 frameworks: OWASP LLM Top 10, Agentic AI Top 10 (ASI), Skills Top 10 (AST), MCP Top 10, AI Agent Traps (DeepMind). 1461 tests.
|
||||
Security scanning, auditing, and threat modeling for Claude Code projects. 5 frameworks: OWASP LLM Top 10, Agentic AI Top 10 (ASI), Skills Top 10 (AST), MCP Top 10, AI Agent Traps (DeepMind). 1485 tests.
|
||||
|
||||
**v7.0.0 — Trustworthy scoring (BREAKING).** Three changes target the false-positive cascade on real codebases (hyperframes.com gave `BLOCK / Extreme / 100`, ~70% noise):
|
||||
|
||||
1. **Risk-score v2 formula** (`scanners/lib/severity.mjs`) — severity-dominated, log-scaled within tier. Replaces v1 sum-and-cap that collapsed every non-trivial scan to 100/Extreme. Tiers: critical → 70–95, high only → 40–65, medium only → 15–35, low only → 1–11. Verdict cutoffs realigned to new bands (BLOCK ≥65, WARNING ≥15).
|
||||
2. **Context-aware entropy scanner** — file-extension skip (`.glsl/.frag/.vert/.shader/.wgsl/.css/.scss/.sass/.less/.svg/.min.*/.map`) + 7 new line-suppression rules (GLSL keywords, CSS-in-JS, inline SVG, ffmpeg `filter_complex`, User-Agent strings, SQL DDL, `throw new Error(\`...\`)`). Configurable via `.llm-security/policy.json` `entropy` section (thresholds, `suppress_extensions`, `suppress_line_patterns`, `suppress_paths`). Envelope `calibration` block reports skip counters + effective thresholds + policy source.
|
||||
3. **DEP typosquat allowlist expansion** — 22 npm + 5 PyPI entries for short-name tools that tripped Levenshtein detection on every modern codebase (`knip`, `oxlint`, `tsx`, `nx`, `rimraf`, `uv`, `ruff`, etc.).
|
||||
|
||||
See `docs/security-hardening-guide.md` §6 for the calibration story.
|
||||
|
||||
## Commands
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
|
||||
*AI-generated: all code produced by Claude Code through dialog-driven development. [Full disclosure →](../../README.md#ai-generated-code-disclosure)*
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -824,6 +824,7 @@ This plugin provides full-stack security hardening (static analysis + supply cha
|
|||
|
||||
| Version | Date | Highlights |
|
||||
|---------|------|------------|
|
||||
| **7.0.0** | 2026-04-19 | **Trustworthy scoring (BREAKING).** Three changes target the false-positive cascade on real codebases (scan of hyperframes.com gave `BLOCK / Extreme / 100` with ~70% noise). **1. Risk-score v2** (`scanners/lib/severity.mjs`) — severity-dominated, log-scaled within tier. Replaces sum-and-cap that collapsed every non-trivial scan to 100/Extreme. Tiers: critical → 70–95, high only → 40–65, medium only → 15–35, low only → 1–11. Verdict cutoffs realigned (BLOCK ≥65, WARNING ≥15) for band co-monotonicity. **2. Context-aware entropy scanner** — file-extension skip (`.glsl/.frag/.vert/.shader/.wgsl/.css/.scss/.sass/.less/.svg/.min.*/.map`) + 7 new line-suppression rules (GLSL keywords, CSS-in-JS templates, inline SVG, ffmpeg `filter_complex`, User-Agent strings, SQL DDL on dedicated lines, `throw new Error(\`...\`)`). Configurable via `.llm-security/policy.json` `entropy` section (thresholds, `suppress_extensions`, `suppress_line_patterns`, `suppress_paths`). Envelope `calibration` block reports skip counters + effective thresholds + policy source. **3. DEP typosquat allowlist expansion** — 22 npm + 5 PyPI entries for short-name tools that tripped Levenshtein on every modern codebase (`knip`, `oxlint`, `tsx`, `nx`, `rimraf`, `uv`, `ruff`, etc.). Synthesizer "Scan Calibration" section + "never override verdict" rule added. Legacy `riskScoreV1()` kept for reference. **CI pipelines with `--fail-on` thresholds may need recalibration.** 1485 tests (was 1461). |
|
||||
| **6.6.0** | 2026-04-18 | **JetBrains/IntelliJ plugin scanning.** `/security ide-scan` now covers JetBrains IDEs (IntelliJ IDEA, PyCharm, GoLand, WebStorm, RubyMine, PhpStorm, CLion, DataGrip, RustRover, Rider, Aqua, Writerside, Android Studio) — Fleet and Toolbox excluded. OS-aware discovery of `~/Library/Application Support/JetBrains/<IDE><version>/plugins/` (macOS), `%APPDATA%\JetBrains\...` (Windows), `~/.config/JetBrains/...` (Linux). Zero-dep parsers for `META-INF/plugin.xml` and `META-INF/MANIFEST.MF` with nested-jar extraction. 7 JetBrains-specific checks: theme-with-code, broad activation (`application-components`), `Premain-Class` instrumentation (HIGH — javaagent retransform), native binaries (`.so`/`.dylib`/`.dll`/`.jnilib`), long `<depends>` chains (supply-chain pressure), typosquat vs top JetBrains plugins, shaded-jar advisory. URL fetch for `plugins.jetbrains.com/plugin/<numericId>-<slug>` + direct `/plugin/download?pluginId=<xmlId>`; metadata resolves numericId → xmlId before download. `.kt`, `.groovy`, `.scala` added to `taint-tracer` code extensions. Reuses existing OS sandbox (`lib/vsix-sandbox.mjs` parameterized via `buildSandboxedWorker(..., workerPath)`). Knowledge: `knowledge/jetbrains-marketplace-api-notes.md`, expanded `knowledge/ide-extension-threat-patterns.md`, seeded `knowledge/top-jetbrains-plugins.json`. 1461 tests (was 1352). |
|
||||
| **6.5.0** | 2026-04-17 | **OS sandbox for `/security ide-scan <url>`.** VSIX fetch + extract now runs in a sub-process wrapped by `sandbox-exec` (macOS) or `bwrap` (Linux), reusing the same primitives proven by the v5.1 git-clone sandbox. Defense-in-depth — even if `lib/zip-extract.mjs` ever has a bypass, the kernel refuses any write outside the per-scan temp directory. New: `lib/vsix-fetch-worker.mjs` (sub-process worker with deterministic JSON-line IPC) and `lib/vsix-sandbox.mjs` (`buildSandboxProfile` / `buildBwrapArgs` / `buildSandboxedWorker` / `runVsixWorker`, 35 s timeout, 1 MB stdout cap). New `scan(target, { useSandbox })` option (default `true` for CLI; tests use `false` since `globalThis.fetch` mocks do not cross processes). Windows fallback: in-process with `meta.warnings` advisory. Envelope `meta.source.sandbox` field: `'sandbox-exec' \| 'bwrap' \| 'none' \| 'in-process'`. 1352 tests (was 1344). |
|
||||
| **6.4.0** | 2026-04-17 | **`/security ide-scan <url>` — pre-install verification.** The IDE extension scanner now accepts URLs and fetches the VSIX before scanning. Supported: VS Code Marketplace (`https://marketplace.visualstudio.com/items?itemName=publisher.name`), OpenVSX (`https://open-vsx.org/extension/publisher/name[/version]`), and direct `.vsix` URLs. New libraries: `lib/vsix-fetch.mjs` (HTTPS-only fetch with 50MB cap, 30s timeout, SHA-256, manual host-whitelisted redirects) and `lib/zip-extract.mjs` (zero-dep ZIP parser, rejects zip-slip / symlinks / absolute paths / drive letters / encrypted entries / ZIP64; caps: 10 000 entries, 500MB uncompressed, 100x expansion ratio, depth 20). Temp dir always cleaned in `try/finally`. Envelope `meta.source` carries `{ type: "url", kind, url, finalUrl, sha256, size, publisher, name, version }`. New knowledge file: `marketplace-api-notes.md`. GitHub repo URLs intentionally not supported (would require a build step). 1344 tests (was 1296). |
|
||||
|
|
|
|||
|
|
@ -147,7 +147,103 @@ attacks but does not eliminate them.
|
|||
|
||||
---
|
||||
|
||||
## 6. Recommended baseline for production
|
||||
## 6. Calibration & false positives (v7.0.0+)
|
||||
|
||||
Security scanners live or die by their signal-to-noise ratio. A scanner that
|
||||
cries "extreme" on every project destroys its own credibility — users learn
|
||||
to ignore findings, and genuine threats slip past. v7.0.0 ships three
|
||||
calibration layers to keep that from happening.
|
||||
|
||||
### 6.1 Risk-score v2 formula
|
||||
|
||||
The v1 formula was a sum-and-cap: `critical*25 + high*10 + medium*4 + low*1`,
|
||||
capped at 100. Every non-trivial scan collapsed to 100/Extreme regardless of
|
||||
actual distribution. A codebase with 2 mediums and 100 lows scored the same
|
||||
as a codebase with 5 criticals.
|
||||
|
||||
v2 (`scanners/lib/severity.mjs`) is severity-dominated and log-scaled within
|
||||
tier:
|
||||
|
||||
| Finding mix | Score range | Band |
|
||||
|-------------|-------------|------|
|
||||
| Critical present | 70–95 (1=80, 2=86, 4=90, 10=95) | Critical/Extreme |
|
||||
| High only | 40–65 (1=48, 5=60, 17=65) | High |
|
||||
| Medium only | 15–35 (1=20, 5=28, 50=33) | Medium |
|
||||
| Low only | 1–11 (1=4, 10=11) | Low |
|
||||
| None | 0 | Low |
|
||||
|
||||
Verdict cutoffs (`BLOCK ≥65`, `WARNING ≥15`) are locked to the `riskBand()`
|
||||
boundaries so you can't get a "BLOCK / Medium band" contradiction. The legacy
|
||||
formula is kept as `riskScoreV1()` for reference only.
|
||||
|
||||
**CI impact:** Pipelines with `--fail-on high` keep working (the severity
|
||||
gate is unaffected). Pipelines with score-based thresholds need recalibration
|
||||
— old `score >= 21` corresponds roughly to new `score >= 15`.
|
||||
|
||||
### 6.2 Context-aware entropy scanner
|
||||
|
||||
The entropy scanner flags high-Shannon-entropy strings as possible
|
||||
credentials. On codebases heavy with shader code, bundled JS, CSS-in-JS or
|
||||
SQL it produced astronomical false-positive rates. v7.0.0 adds three
|
||||
suppression layers:
|
||||
|
||||
1. **File-extension skip** — whole files with these extensions are never
|
||||
inspected for entropy findings: `.glsl, .frag, .vert, .shader, .wgsl,
|
||||
.css, .scss, .sass, .less, .svg` + compound `.min.js, .min.css, .map`. A
|
||||
skip counter (`calibration.files_skipped_by_extension`) is reported in the
|
||||
scanner envelope.
|
||||
2. **Line-level rules 11–17** — applied when a line contains any of: GLSL
|
||||
keywords (`uniform`, `vec3`, `texture2D`…), CSS-in-JS templates
|
||||
(`styled.…`), inline `<svg>` markup, ffmpeg `filter_complex` syntax,
|
||||
browser `User-Agent` strings, SQL DDL on a dedicated line
|
||||
(`^\s*(SELECT|INSERT|…)`), or `throw new Error(\`…\`)` templates.
|
||||
3. **Per-project policy override** — `.llm-security/policy.json` `entropy`
|
||||
section supports:
|
||||
|
||||
```json
|
||||
{
|
||||
"entropy": {
|
||||
"thresholds": {
|
||||
"critical": { "entropy": 5.4, "minLen": 128 },
|
||||
"high": { "entropy": 5.1, "minLen": 64 },
|
||||
"medium": { "entropy": 4.7, "minLen": 40 }
|
||||
},
|
||||
"suppress_extensions": [".custom"],
|
||||
"suppress_line_patterns": ["MY_VENDOR_MARKER"],
|
||||
"suppress_paths": ["vendored/", "generated/"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The synthesizer agent reports calibration prominently if >80 % of files were
|
||||
skipped (signals a policy so aggressive the scan is effectively bypassed)
|
||||
and omits it silently if <5 % were skipped.
|
||||
|
||||
### 6.3 Typosquat allowlist
|
||||
|
||||
The DEP scanner flags Levenshtein-close package names against a top-N list
|
||||
to catch typosquats (`lod-ash`, `expres`). On real codebases this tripped on
|
||||
short-name tools like `knip`, `nx`, `tsx`, `uv`, `ruff`. v7.0.0 extends
|
||||
`knowledge/typosquat-allowlist.json` with 22 npm + 5 PyPI entries for modern
|
||||
tools.
|
||||
|
||||
### 6.4 Tuning workflow
|
||||
|
||||
1. Run `/security deep-scan` on a representative codebase.
|
||||
2. Read `calibration.files_skipped_by_extension` and `files_skipped_by_path`
|
||||
from the envelope — are they reasonable?
|
||||
3. Review the top 10 findings. For each false positive, pick the narrowest
|
||||
suppression that catches it:
|
||||
- Whole extension noisy → `suppress_extensions`
|
||||
- One line pattern recurring → `suppress_line_patterns`
|
||||
- Whole directory vendored → `suppress_paths`
|
||||
4. Raise thresholds only as a last resort — you're hiding real signal.
|
||||
5. Re-scan and verify verdict/band/score make sense relative to the finding
|
||||
set.
|
||||
|
||||
---
|
||||
|
||||
## 7. Recommended baseline for production
|
||||
|
||||
1. Set `CLAUDE_CODE_EFFORT_LEVEL=xhigh` for audit and planning sessions.
|
||||
2. Set `ENABLE_PROMPT_CACHING_1H=1` globally — reduces cost, does not weaken
|
||||
|
|
@ -155,9 +251,11 @@ attacks but does not eliminate them.
|
|||
3. All three plugin hook modes: start at `warn`, promote to `block` after
|
||||
baselining.
|
||||
4. Keep sandbox wrappers enabled (default on macOS / Linux).
|
||||
5. Periodically run `/security posture` (13-category scorecard) and
|
||||
5. Periodically run `/security posture` (16-category scorecard) and
|
||||
`/security dashboard` (cross-project view) to catch drift.
|
||||
6. After first `/security deep-scan`, run the §6.4 tuning workflow once to
|
||||
calibrate the noise floor for your codebase.
|
||||
|
||||
---
|
||||
|
||||
**Last updated:** 2026-04-17 for v6.2.0.
|
||||
**Last updated:** 2026-04-19 for v7.0.0.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "llm-security",
|
||||
"version": "6.6.0",
|
||||
"version": "7.0.0",
|
||||
"description": "Security scanning, auditing, and threat modeling for Claude Code projects",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
|
|
|
|||
|
|
@ -10,6 +10,8 @@
|
|||
// - OWASP LLM03 (Supply Chain — obfuscated dependencies)
|
||||
// - ToxicSkills research: evasion via base64-wrapped instructions
|
||||
|
||||
import { existsSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { readTextFile } from './lib/file-discovery.mjs';
|
||||
import { finding, scannerResult } from './lib/output.mjs';
|
||||
import { SEVERITY } from './lib/severity.mjs';
|
||||
|
|
@ -436,9 +438,15 @@ export async function scan(targetPath, discovery) {
|
|||
let filesScanned = 0;
|
||||
|
||||
// Load policy for this target and apply overrides to module-level state.
|
||||
// Best-effort — on any error we fall back to built-in defaults.
|
||||
// Best-effort — on any error we fall back to built-in defaults. Provenance
|
||||
// tracked via file-existence check, not by comparing merged values (defaults
|
||||
// always include an entropy section so a value-based check would always
|
||||
// report 'policy.json').
|
||||
let policySource = 'defaults';
|
||||
try {
|
||||
if (existsSync(join(targetPath, '.llm-security', 'policy.json'))) {
|
||||
policySource = 'policy.json';
|
||||
}
|
||||
const policy = loadPolicy(targetPath);
|
||||
const ent = policy?.entropy || {};
|
||||
THRESHOLDS = resolveThresholds(ent.thresholds);
|
||||
|
|
@ -449,19 +457,12 @@ export async function scan(targetPath, discovery) {
|
|||
.filter((e) => typeof e === 'string')
|
||||
.map((e) => e.toLowerCase()),
|
||||
);
|
||||
if (
|
||||
ent.thresholds ||
|
||||
(ent.suppress_line_patterns && ent.suppress_line_patterns.length > 0) ||
|
||||
(ent.suppress_paths && ent.suppress_paths.length > 0) ||
|
||||
(ent.suppress_extensions && ent.suppress_extensions.length > 0)
|
||||
) {
|
||||
policySource = 'policy.json';
|
||||
}
|
||||
} catch {
|
||||
THRESHOLDS = DEFAULT_THRESHOLDS;
|
||||
USER_SUPPRESS_LINE_PATTERNS = [];
|
||||
USER_SUPPRESS_PATHS = [];
|
||||
USER_SUPPRESS_EXTENSIONS = new Set();
|
||||
policySource = 'defaults';
|
||||
}
|
||||
|
||||
let filesSkippedByExtension = 0;
|
||||
|
|
|
|||
|
|
@ -49,7 +49,7 @@ import { scan as scanTaint } from './taint-tracer.mjs';
|
|||
import { scan as scanMemoryPoisoning } from './memory-poisoning-scanner.mjs';
|
||||
import { scan as scanSupplyChain } from './supply-chain-recheck.mjs';
|
||||
|
||||
const VERSION = '6.6.0';
|
||||
const VERSION = '7.0.0';
|
||||
const SCANNER = 'IDE';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
|
|
|||
|
|
@ -16,7 +16,8 @@ const TEXT_EXTENSIONS = new Set([
|
|||
'.env', '.env.local', '.env.example',
|
||||
'.cfg', '.ini', '.conf',
|
||||
'.xml', '.html', '.htm', '.svg',
|
||||
'.css', '.scss', '.less',
|
||||
'.css', '.scss', '.sass', '.less',
|
||||
'.glsl', '.frag', '.vert', '.shader', '.wgsl', // GPU shader source
|
||||
'.sql',
|
||||
'.rs', '.go', '.java', '.kt', '.cs', '.c', '.cpp', '.h', '.hpp',
|
||||
'.rb', '.php', '.lua', '.swift', '.m',
|
||||
|
|
|
|||
|
|
@ -63,15 +63,18 @@ export function riskScoreV1(counts) {
|
|||
}
|
||||
|
||||
/**
|
||||
* Derive verdict from severity counts and risk score.
|
||||
* BLOCK if Critical >= 1 OR score >= 61. WARNING if High >= 1 OR score >= 21. Otherwise ALLOW.
|
||||
* Derive verdict from severity counts and risk score (v7.0.0 thresholds).
|
||||
* Aligned to v2 riskBand cutoffs so verdict and band are co-monotonic:
|
||||
* BLOCK if critical >= 1 OR score >= 65 (Critical/Extreme band)
|
||||
* WARNING if high >= 1 OR score >= 15 (Medium/High band)
|
||||
* ALLOW otherwise (Low band)
|
||||
* @param {{ critical: number, high: number, medium: number, low: number, info: number }} counts
|
||||
* @returns {'BLOCK' | 'WARNING' | 'ALLOW'}
|
||||
*/
|
||||
export function verdict(counts) {
|
||||
const score = riskScore(counts);
|
||||
if ((counts.critical || 0) >= 1 || score >= 61) return 'BLOCK';
|
||||
if ((counts.high || 0) >= 1 || score >= 21) return 'WARNING';
|
||||
if ((counts.critical || 0) >= 1 || score >= 65) return 'BLOCK';
|
||||
if ((counts.high || 0) >= 1 || score >= 15) return 'WARNING';
|
||||
return 'ALLOW';
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -234,13 +234,13 @@ describe('envelope', () => {
|
|||
});
|
||||
|
||||
it('computes correct risk_score from aggregated counts', () => {
|
||||
// 1 critical = score 25
|
||||
// v2 formula (v7.0.0+): 1 critical = score 80 (70 + log2(2)*10 = 80)
|
||||
const f = finding({ scanner: 'ENT', severity: 'critical', title: 'C', description: 'x' });
|
||||
const scanners = {
|
||||
entropy: scannerResult('entropy-scanner', 'ok', [f], 5, 30),
|
||||
};
|
||||
const result = envelope('/project', scanners, 30);
|
||||
assert.equal(result.aggregate.risk_score, 25);
|
||||
assert.equal(result.aggregate.risk_score, 80);
|
||||
});
|
||||
|
||||
it('returns BLOCK verdict when critical finding present', () => {
|
||||
|
|
|
|||
|
|
@ -46,7 +46,7 @@ describe('SEVERITY', () => {
|
|||
// riskScore
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('riskScore', () => {
|
||||
describe('riskScore (v2 — severity-dominated log-scaled, v7.0.0+)', () => {
|
||||
it('returns 0 when all counts are zero', () => {
|
||||
assert.equal(riskScore({ critical: 0, high: 0, medium: 0, low: 0, info: 0 }), 0);
|
||||
});
|
||||
|
|
@ -55,37 +55,75 @@ describe('riskScore', () => {
|
|||
assert.equal(riskScore({}), 0);
|
||||
});
|
||||
|
||||
it('returns 25 for one critical finding (weight=25)', () => {
|
||||
assert.equal(riskScore({ critical: 1 }), 25);
|
||||
});
|
||||
|
||||
it('returns 100 (capped) for four critical findings (4*25=100)', () => {
|
||||
assert.equal(riskScore({ critical: 4 }), 100);
|
||||
});
|
||||
|
||||
it('caps at 100 even if raw score would exceed it', () => {
|
||||
assert.equal(riskScore({ critical: 10, high: 10 }), 100);
|
||||
});
|
||||
|
||||
it('returns 10 for one high finding (weight=10)', () => {
|
||||
assert.equal(riskScore({ high: 1 }), 10);
|
||||
});
|
||||
|
||||
it('returns 4 for one medium finding (weight=4)', () => {
|
||||
assert.equal(riskScore({ medium: 1 }), 4);
|
||||
});
|
||||
|
||||
it('returns 1 for one low finding (weight=1)', () => {
|
||||
assert.equal(riskScore({ low: 1 }), 1);
|
||||
});
|
||||
|
||||
it('returns 0 for info-only findings (weight=0)', () => {
|
||||
it('returns 0 for info-only findings (info tier is non-scoring)', () => {
|
||||
assert.equal(riskScore({ info: 100 }), 0);
|
||||
});
|
||||
|
||||
it('returns correct sum for mixed counts', () => {
|
||||
// 1*25 + 2*10 + 3*4 + 4*1 + 5*0 = 25+20+12+4+0 = 61
|
||||
assert.equal(riskScore({ critical: 1, high: 2, medium: 3, low: 4, info: 5 }), 61);
|
||||
// --- Low tier: 1 + log2(n+1)*3, capped at 11 ---
|
||||
it('returns 4 for one low finding', () => {
|
||||
assert.equal(riskScore({ low: 1 }), 4);
|
||||
});
|
||||
|
||||
it('returns 11 for twenty low findings (tier-capped)', () => {
|
||||
assert.equal(riskScore({ low: 20 }), 11);
|
||||
});
|
||||
|
||||
// --- Medium tier: 15 + log2(n+1)*5, capped at 35 ---
|
||||
it('returns 20 for one medium finding (tier base + log scale)', () => {
|
||||
assert.equal(riskScore({ medium: 1 }), 20);
|
||||
});
|
||||
|
||||
it('returns 28 for five medium findings', () => {
|
||||
assert.equal(riskScore({ medium: 5 }), 28);
|
||||
});
|
||||
|
||||
it('returns 29 for six medium findings (still inside Medium band)', () => {
|
||||
assert.equal(riskScore({ medium: 6 }), 29);
|
||||
});
|
||||
|
||||
// --- High tier: 40 + log2(n+1)*8, capped at 65 ---
|
||||
it('returns 48 for one high finding', () => {
|
||||
assert.equal(riskScore({ high: 1 }), 48);
|
||||
});
|
||||
|
||||
it('returns 64 for seven high findings (just below Critical band)', () => {
|
||||
assert.equal(riskScore({ high: 7 }), 64);
|
||||
});
|
||||
|
||||
it('returns 65 when high tier saturates — many high + many medium', () => {
|
||||
// 17 high + 136 medium (hyperframes-like) → high-tier dominates, cap 65
|
||||
assert.equal(riskScore({ high: 17, medium: 136 }), 65);
|
||||
});
|
||||
|
||||
// --- Critical tier: 70 + log2(n+1)*10, capped at 95 ---
|
||||
it('returns 80 for one critical finding', () => {
|
||||
assert.equal(riskScore({ critical: 1 }), 80);
|
||||
});
|
||||
|
||||
it('returns 86 for two critical findings (enters Extreme band)', () => {
|
||||
assert.equal(riskScore({ critical: 2 }), 86);
|
||||
});
|
||||
|
||||
it('returns 93 for four critical findings', () => {
|
||||
assert.equal(riskScore({ critical: 4 }), 93);
|
||||
});
|
||||
|
||||
it('returns 95 for ten critical findings (tier-capped)', () => {
|
||||
assert.equal(riskScore({ critical: 10 }), 95);
|
||||
});
|
||||
|
||||
it('does not exceed 100 even with huge critical counts', () => {
|
||||
assert.ok(riskScore({ critical: 1000 }) <= 100);
|
||||
});
|
||||
|
||||
it('critical dominates high — mixed critical+high scored at critical tier', () => {
|
||||
// {critical:1, high:2} → critical tier: 70 + log2(2)*10 = 80
|
||||
assert.equal(riskScore({ critical: 1, high: 2, medium: 3, low: 4, info: 5 }), 80);
|
||||
});
|
||||
|
||||
it('high dominates medium — {high:1, medium:100} scored at high tier', () => {
|
||||
// 40 + log2(2)*8 = 48
|
||||
assert.equal(riskScore({ high: 1, medium: 100 }), 48);
|
||||
});
|
||||
});
|
||||
|
||||
|
|
@ -93,7 +131,7 @@ describe('riskScore', () => {
|
|||
// verdict
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('verdict', () => {
|
||||
describe('verdict (v7.0.0 — co-monotonic with riskBand)', () => {
|
||||
it('returns ALLOW for zero findings', () => {
|
||||
assert.equal(verdict({ critical: 0, high: 0, medium: 0, low: 0, info: 0 }), 'ALLOW');
|
||||
});
|
||||
|
|
@ -102,37 +140,36 @@ describe('verdict', () => {
|
|||
assert.equal(verdict({}), 'ALLOW');
|
||||
});
|
||||
|
||||
it('returns BLOCK when critical >= 1', () => {
|
||||
it('returns BLOCK when critical >= 1 (score=80)', () => {
|
||||
assert.equal(verdict({ critical: 1 }), 'BLOCK');
|
||||
});
|
||||
|
||||
it('returns BLOCK when score >= 61 (even with no critical)', () => {
|
||||
// Need score >= 61 without critical: 7 high = 70 >= 61
|
||||
assert.equal(verdict({ high: 7 }), 'BLOCK');
|
||||
it('returns BLOCK when score >= 65 without critical (17 high + 136 medium = 65)', () => {
|
||||
assert.equal(verdict({ high: 17, medium: 136 }), 'BLOCK');
|
||||
});
|
||||
|
||||
it('returns BLOCK for score exactly 61', () => {
|
||||
// 1 critical + 2 high + 3 medium + 4 low = 25+20+12+4 = 61
|
||||
assert.equal(verdict({ critical: 1, high: 2, medium: 3, low: 4 }), 'BLOCK');
|
||||
it('returns WARNING for 7 high findings (score=64, Critical band boundary not crossed)', () => {
|
||||
assert.equal(verdict({ high: 7 }), 'WARNING');
|
||||
});
|
||||
|
||||
it('returns WARNING when high >= 1 (and no critical)', () => {
|
||||
assert.equal(verdict({ high: 1 }), 'WARNING');
|
||||
});
|
||||
|
||||
it('returns WARNING when score >= 21 (even with no high or critical)', () => {
|
||||
// 6 medium = 24 >= 21; no critical or high
|
||||
it('returns WARNING for 1 medium (score=20, inside Medium band)', () => {
|
||||
assert.equal(verdict({ medium: 1 }), 'WARNING');
|
||||
});
|
||||
|
||||
it('returns WARNING for 6 medium (score=29)', () => {
|
||||
assert.equal(verdict({ medium: 6 }), 'WARNING');
|
||||
});
|
||||
|
||||
it('returns WARNING for score exactly 21 (no high or critical)', () => {
|
||||
// Smallest score >= 21 from low only would need 21 low, but medium is easier:
|
||||
// 5 medium + 1 low = 20+1 = 21
|
||||
assert.equal(verdict({ medium: 5, low: 1 }), 'WARNING');
|
||||
it('returns ALLOW for 20 low findings (score=11, firmly Low band)', () => {
|
||||
assert.equal(verdict({ low: 20 }), 'ALLOW');
|
||||
});
|
||||
|
||||
it('returns ALLOW for score of 20 (low only, no high/critical)', () => {
|
||||
assert.equal(verdict({ low: 20 }), 'ALLOW');
|
||||
it('returns ALLOW for 1 low finding (score=4)', () => {
|
||||
assert.equal(verdict({ low: 1 }), 'ALLOW');
|
||||
});
|
||||
});
|
||||
|
||||
|
|
@ -140,56 +177,56 @@ describe('verdict', () => {
|
|||
// riskBand
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('riskBand', () => {
|
||||
describe('riskBand (v7.0.0 cutoffs: 14/39/64/84)', () => {
|
||||
it('returns Low for score 0', () => {
|
||||
assert.equal(riskBand(0), 'Low');
|
||||
});
|
||||
|
||||
it('returns Low for score 20 (boundary)', () => {
|
||||
assert.equal(riskBand(20), 'Low');
|
||||
it('returns Low for score 14 (upper boundary)', () => {
|
||||
assert.equal(riskBand(14), 'Low');
|
||||
});
|
||||
|
||||
it('returns Medium for score 21', () => {
|
||||
assert.equal(riskBand(21), 'Medium');
|
||||
it('returns Medium for score 15 (Medium tier start)', () => {
|
||||
assert.equal(riskBand(15), 'Medium');
|
||||
});
|
||||
|
||||
it('returns Medium for score 25', () => {
|
||||
assert.equal(riskBand(25), 'Medium');
|
||||
it('returns Medium for score 20 (one medium finding)', () => {
|
||||
assert.equal(riskBand(20), 'Medium');
|
||||
});
|
||||
|
||||
it('returns Medium for score 40 (boundary)', () => {
|
||||
assert.equal(riskBand(40), 'Medium');
|
||||
it('returns Medium for score 39 (upper boundary)', () => {
|
||||
assert.equal(riskBand(39), 'Medium');
|
||||
});
|
||||
|
||||
it('returns High for score 41', () => {
|
||||
assert.equal(riskBand(41), 'High');
|
||||
it('returns High for score 40 (High tier start — one high finding is 48)', () => {
|
||||
assert.equal(riskBand(40), 'High');
|
||||
});
|
||||
|
||||
it('returns High for score 50', () => {
|
||||
assert.equal(riskBand(50), 'High');
|
||||
it('returns High for score 48 (one high finding)', () => {
|
||||
assert.equal(riskBand(48), 'High');
|
||||
});
|
||||
|
||||
it('returns High for score 60 (boundary)', () => {
|
||||
assert.equal(riskBand(60), 'High');
|
||||
it('returns High for score 64 (seven high findings, upper boundary)', () => {
|
||||
assert.equal(riskBand(64), 'High');
|
||||
});
|
||||
|
||||
it('returns Critical for score 61', () => {
|
||||
assert.equal(riskBand(61), 'Critical');
|
||||
it('returns Critical for score 65 (many high without critical)', () => {
|
||||
assert.equal(riskBand(65), 'Critical');
|
||||
});
|
||||
|
||||
it('returns Critical for score 75', () => {
|
||||
assert.equal(riskBand(75), 'Critical');
|
||||
});
|
||||
|
||||
it('returns Critical for score 80 (boundary)', () => {
|
||||
it('returns Critical for score 80 (one critical finding)', () => {
|
||||
assert.equal(riskBand(80), 'Critical');
|
||||
});
|
||||
|
||||
it('returns Extreme for score 81', () => {
|
||||
assert.equal(riskBand(81), 'Extreme');
|
||||
it('returns Critical for score 84 (upper boundary)', () => {
|
||||
assert.equal(riskBand(84), 'Critical');
|
||||
});
|
||||
|
||||
it('returns Extreme for score 95', () => {
|
||||
it('returns Extreme for score 85 (two critical findings reach here)', () => {
|
||||
assert.equal(riskBand(85), 'Extreme');
|
||||
});
|
||||
|
||||
it('returns Extreme for score 95 (ten critical findings, tier-capped)', () => {
|
||||
assert.equal(riskBand(95), 'Extreme');
|
||||
});
|
||||
|
||||
|
|
|
|||
243
plugins/llm-security/tests/scanners/entropy-context.test.mjs
Normal file
243
plugins/llm-security/tests/scanners/entropy-context.test.mjs
Normal file
|
|
@ -0,0 +1,243 @@
|
|||
// entropy-context.test.mjs — False-positive fixtures for v7.0.0 context-aware suppression
|
||||
//
|
||||
// Covers:
|
||||
// A. File-extension skip (.glsl, .css, .svg, .min.js, ...)
|
||||
// B. Line-level rules 11-17 (GLSL/CSS-in-JS/HTML/ffmpeg/UA/SQL/error-template)
|
||||
// C. User-policy thresholds and suppress_line_patterns
|
||||
//
|
||||
// Strategy: write a throwaway fixture under os.tmpdir(), discover it, run scan(),
|
||||
// assert finding count. Fixture-content strings are built from runtime concatenation
|
||||
// to avoid triggering the plugin's own credential-pattern pre-edit hook on the test source.
|
||||
|
||||
import { describe, it, before, after } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { randomBytes } from 'node:crypto';
|
||||
import { join } from 'node:path';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { discoverFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { scan } from '../../scanners/entropy-scanner.mjs';
|
||||
import { _resetCacheForTest } from '../../scanners/lib/policy-loader.mjs';
|
||||
|
||||
// Random base64 from 60 crypto bytes → 80-char base64, H ≈ 5.4, will classify as
|
||||
// HIGH (entropy >= 5.1, len >= 64). Regenerated on module load for each test run.
|
||||
// Built at runtime so the plugin's credential-pattern pre-edit hook doesn't flag the
|
||||
// test source file. Excludes '/', '+', '=' to avoid breaking JS string syntax.
|
||||
function makePayload() {
|
||||
const raw = randomBytes(60).toString('base64').replace(/[/+=]/g, 'A');
|
||||
return raw.slice(0, 80);
|
||||
}
|
||||
const PAYLOAD = makePayload();
|
||||
|
||||
async function writeFixture(root, relPath, content) {
|
||||
const abs = join(root, relPath);
|
||||
const lastSlash = abs.lastIndexOf('/');
|
||||
await mkdir(abs.substring(0, lastSlash), { recursive: true });
|
||||
await writeFile(abs, content);
|
||||
}
|
||||
|
||||
async function newRoot(prefix) {
|
||||
return mkdtemp(join(tmpdir(), prefix));
|
||||
}
|
||||
|
||||
describe('entropy-scanner context suppression (v7.0.0+)', () => {
|
||||
let root;
|
||||
|
||||
before(async () => {
|
||||
root = await newRoot('entropy-ctx-');
|
||||
});
|
||||
|
||||
after(async () => {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
_resetCacheForTest();
|
||||
});
|
||||
|
||||
describe('A. File-extension skip', () => {
|
||||
it('skips .glsl files entirely (no findings)', async () => {
|
||||
const fx = await newRoot('ent-glsl-');
|
||||
await writeFixture(fx, 'shader.glsl', 'vec4 color = "' + PAYLOAD + '";');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'Expected 0 findings in .glsl, got ' + result.findings.length);
|
||||
assert.ok(result.calibration.files_skipped_by_extension >= 1);
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('skips .css files entirely', async () => {
|
||||
const fx = await newRoot('ent-css-');
|
||||
await writeFixture(fx, 'styles.css', '.x{content:"' + PAYLOAD + '";}');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0);
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('skips .min.js files (compound extension)', async () => {
|
||||
const fx = await newRoot('ent-minjs-');
|
||||
await writeFixture(fx, 'bundle.min.js', 'var x="' + PAYLOAD + '";');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0);
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('still scans .js files (non-skipped extension)', async () => {
|
||||
const fx = await newRoot('ent-js-');
|
||||
await writeFixture(fx, 'app.js', 'const blob = "' + PAYLOAD + '";');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.ok(result.findings.length >= 1, 'expected high-entropy .js to still be detected');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('B. Line-level suppression rules 11-17', () => {
|
||||
it('rule 11: GLSL keyword on line suppresses finding', async () => {
|
||||
const fx = await newRoot('ent-rule11-');
|
||||
await writeFixture(fx, 'shader.js',
|
||||
'const src = "uniform vec3 u_resolution; ' + PAYLOAD + '";');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected GLSL keyword line to suppress');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rule 12: CSS-in-JS (styled-components) suppresses finding', async () => {
|
||||
const fx = await newRoot('ent-rule12-');
|
||||
await writeFixture(fx, 'btn.js',
|
||||
'const Button = styled.button`:hover { content: "' + PAYLOAD + '"; }`;');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected styled-components line to suppress');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rule 13: Inline <svg> markup on line suppresses finding', async () => {
|
||||
const fx = await newRoot('ent-rule13-');
|
||||
await writeFixture(fx, 'Icon.jsx',
|
||||
'return (<svg viewBox="0 0 24 24"><path d="' + PAYLOAD + '"/></svg>);');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected inline SVG line to suppress');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rule 14: ffmpeg filter_complex suppresses finding', async () => {
|
||||
const fx = await newRoot('ent-rule14-');
|
||||
await writeFixture(fx, 'pipeline.js',
|
||||
'run("ffmpeg -filter_complex=[0:v]scale=' + PAYLOAD + '");');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected ffmpeg line to suppress');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rule 15: browser User-Agent string suppresses finding', async () => {
|
||||
const fx = await newRoot('ent-rule15-');
|
||||
await writeFixture(fx, 'ua.js',
|
||||
'const agent = "Mozilla/5.0 Chrome/120 ' + PAYLOAD + '";');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected UA line to suppress');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rule 16: SQL DDL on dedicated line suppresses finding', async () => {
|
||||
// Line must START with SELECT/INSERT/... — whitespace allowed but no prefix code.
|
||||
const fx = await newRoot('ent-rule16-');
|
||||
await writeFixture(fx, 'schema.js',
|
||||
'// fallback\nSELECT id, data FROM users WHERE tok = \'' + PAYLOAD + '\';\n');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected SELECT-anchored line to suppress');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rule 16 does NOT over-match generic strings mentioning SELECT', async () => {
|
||||
// SQL_STATEMENT is line-anchored; a `const` prefix means no suppression by rule 16.
|
||||
const fx = await newRoot('ent-rule16b-');
|
||||
await writeFixture(fx, 'app.js',
|
||||
'const msg = "SELECT something ' + PAYLOAD + '";');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.ok(result.findings.length >= 1, 'generic code line must not trigger SQL suppression');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rule 17: throw new Error template suppresses finding', async () => {
|
||||
const fx = await newRoot('ent-rule17-');
|
||||
await writeFixture(fx, 'err.js',
|
||||
'throw new Error(`Bad input <code>' + PAYLOAD + '</code>`);');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected throw new Error line to suppress');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('C. Policy-driven overrides', () => {
|
||||
it('user-policy suppress_line_patterns adds custom suppression', async () => {
|
||||
const fx = await newRoot('ent-policy-');
|
||||
await writeFixture(fx, 'secret.js', 'const vendor = "' + PAYLOAD + '"; // MY_VENDOR_MARKER');
|
||||
await writeFixture(fx, '.llm-security/policy.json', JSON.stringify({
|
||||
entropy: { suppress_line_patterns: ['MY_VENDOR_MARKER'] }
|
||||
}));
|
||||
resetCounter();
|
||||
_resetCacheForTest();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected user pattern to suppress');
|
||||
assert.equal(result.calibration.policy_source, 'policy.json');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('user-policy suppress_paths skips files whose relPath contains the substring', async () => {
|
||||
const fx = await newRoot('ent-paths-');
|
||||
await writeFixture(fx, 'src/vendored/big.js', 'var x="' + PAYLOAD + '";');
|
||||
await writeFixture(fx, 'src/app.js', 'var y="' + PAYLOAD + '";');
|
||||
await writeFixture(fx, '.llm-security/policy.json', JSON.stringify({
|
||||
entropy: { suppress_paths: ['vendored/'] }
|
||||
}));
|
||||
resetCounter();
|
||||
_resetCacheForTest();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 1, 'Expected 1 finding (app.js only), got ' + result.findings.length);
|
||||
assert.ok(result.calibration.files_skipped_by_path >= 1);
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('user-policy stricter thresholds suppress medium-strength payload', async () => {
|
||||
const fx = await newRoot('ent-thresh-');
|
||||
await writeFixture(fx, 'cfg.js', 'const blob = "' + PAYLOAD + '";');
|
||||
await writeFixture(fx, '.llm-security/policy.json', JSON.stringify({
|
||||
entropy: {
|
||||
thresholds: {
|
||||
critical: { entropy: 6.0, minLen: 256 },
|
||||
high: { entropy: 5.8, minLen: 200 },
|
||||
medium: { entropy: 5.7, minLen: 150 },
|
||||
}
|
||||
}
|
||||
}));
|
||||
resetCounter();
|
||||
_resetCacheForTest();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected strict thresholds to suppress medium-strength payload');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue