feat(llm-security): v7.0.0 commit 7 — rule 18 (markdown image URL suppression)
E2E verification against content-heavy repo (`content-claude-code`) revealed 413 entropy findings (8 HIGH / 405 MEDIUM) from markdown image CDN URLs in JSON content indexes — e.g., ``. These are legitimate content-repo artifacts, not credentials. The 40-char hash segment in the CDN URL trips Shannon entropy (H=5.29 over 300 chars), and rule 13 (inline <svg>) doesn't match since there's no literal `<svg>` tag — the `.svg` is just a URL path suffix. Added rule 18 `MARKDOWN_IMAGE = /!\[[^\]]*\]\(\s*https?:\/\//` — matches `` / ``. Line-level (not string-level) so URL is not over-specific. E2E impact on `content-claude-code`: - Before: BLOCK / 65 / 8H 437M 0L - After: WARNING / 56 / 3H 427M 0L Hyperframes unchanged: BLOCK / 80 / 1C 4H 92M — real CRITICAL SQL-injection and HIGH findings still detected. Tests: 2 new (positive + negative fixture) bringing entropy-context to 26, total suite 1485 → 1487. Docs updated to "rules 11-18" and "8 new line-suppression rules". Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen
parent
5a4f29fd14
commit
765bc74f52
7 changed files with 43 additions and 9 deletions
|
|
@ -19,7 +19,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|||
### Added
|
||||
- **Context-aware entropy scanner** (`scanners/entropy-scanner.mjs`). Skip-lists and line-level rules drastically reduce false positives in shader/CSS/HTML/SQL-heavy codebases:
|
||||
- File-extension skip: `.glsl, .frag, .vert, .shader, .wgsl, .css, .scss, .sass, .less, .svg` + compound `.min.js, .min.css, .map`
|
||||
- Line-level rules 11–17 in `isFalsePositive()`: GLSL keywords (`uniform`, `vec3`, `texture2D`...), CSS-in-JS templates (`styled.`), inline `<svg>` markup, ffmpeg `filter_complex` syntax, browser `User-Agent` strings, SQL DDL on dedicated lines (`^\s*(SELECT|INSERT|UPDATE|DELETE|CREATE|...)`), `throw new Error(\`…\`)` templates
|
||||
- Line-level rules 11–18 in `isFalsePositive()`: GLSL keywords (`uniform`, `vec3`, `texture2D`...), CSS-in-JS templates (`styled.`), inline `<svg>` markup, ffmpeg `filter_complex` syntax, browser `User-Agent` strings, SQL DDL on dedicated lines (`^\s*(SELECT|INSERT|UPDATE|DELETE|CREATE|...)`), `throw new Error(\`…\`)` templates, markdown image syntax with external URLs (`` — common in JSON content indexes)
|
||||
- Scanner envelope gains `calibration` block: `files_skipped_by_extension`, `files_skipped_by_path`, effective `thresholds`, and `policy_source` (`'default' | 'policy.json'`)
|
||||
- **Policy-driven entropy configuration** — `.llm-security/policy.json` `entropy` section accepts:
|
||||
- `thresholds.{critical,high,medium}.{entropy,minLen}` — override defaults per project
|
||||
|
|
@ -30,7 +30,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|||
- npm: `knip`, `oxlint`, `tsx`, `nx`, `rimraf`, `glob`, `tar`, `zod`, `ky`, `ow`, `esm`, `ip`, `qs`, `url`, `prettier`, `vitest`, `vite`, `rollup`, `swc`, `turbo`, `bun`, `deno`
|
||||
- PyPI: `uv`, `ruff`, `rich`, `typer`, `anyio`
|
||||
- **Synthesizer "Scan Calibration" section** (`agents/deep-scan-synthesizer-agent.md`). Heuristic: omit if <5% files skipped, flag prominently if >80% skipped by path (signals over-aggressive user policy). Agent instructed to NEVER override scanner verdict with narrative opinion.
|
||||
- **24 new unit tests** (`tests/scanners/entropy-context.test.mjs`): A. File-extension skip (4), B. Line-level rules 11–17 (8), C. Policy overrides (3); plus expanded `tests/lib/severity.test.mjs` with v2 scoring/band/verdict tables (70 tests total, was 52). **Total: 1485 tests (was 1461).**
|
||||
- **26 new unit tests** (`tests/scanners/entropy-context.test.mjs`): A. File-extension skip (4), B. Line-level rules 11–18 (10), C. Policy overrides (3); plus expanded `tests/lib/severity.test.mjs` with v2 scoring/band/verdict tables (70 tests total, was 52). **Total: 1487 tests (was 1461).**
|
||||
|
||||
### Changed
|
||||
- `tests/lib/output.test.mjs:243` — "1 critical = score 80" under v2 (was 25 under v1).
|
||||
|
|
|
|||
|
|
@ -1,11 +1,11 @@
|
|||
# LLM Security Plugin (v7.0.0)
|
||||
|
||||
Security scanning, auditing, and threat modeling for Claude Code projects. 5 frameworks: OWASP LLM Top 10, Agentic AI Top 10 (ASI), Skills Top 10 (AST), MCP Top 10, AI Agent Traps (DeepMind). 1485 tests.
|
||||
Security scanning, auditing, and threat modeling for Claude Code projects. 5 frameworks: OWASP LLM Top 10, Agentic AI Top 10 (ASI), Skills Top 10 (AST), MCP Top 10, AI Agent Traps (DeepMind). 1487 tests.
|
||||
|
||||
**v7.0.0 — Trustworthy scoring (BREAKING).** Three changes target the false-positive cascade on real codebases (hyperframes.com gave `BLOCK / Extreme / 100`, ~70% noise):
|
||||
|
||||
1. **Risk-score v2 formula** (`scanners/lib/severity.mjs`) — severity-dominated, log-scaled within tier. Replaces v1 sum-and-cap that collapsed every non-trivial scan to 100/Extreme. Tiers: critical → 70–95, high only → 40–65, medium only → 15–35, low only → 1–11. Verdict cutoffs realigned to new bands (BLOCK ≥65, WARNING ≥15).
|
||||
2. **Context-aware entropy scanner** — file-extension skip (`.glsl/.frag/.vert/.shader/.wgsl/.css/.scss/.sass/.less/.svg/.min.*/.map`) + 7 new line-suppression rules (GLSL keywords, CSS-in-JS, inline SVG, ffmpeg `filter_complex`, User-Agent strings, SQL DDL, `throw new Error(\`...\`)`). Configurable via `.llm-security/policy.json` `entropy` section (thresholds, `suppress_extensions`, `suppress_line_patterns`, `suppress_paths`). Envelope `calibration` block reports skip counters + effective thresholds + policy source.
|
||||
2. **Context-aware entropy scanner** — file-extension skip (`.glsl/.frag/.vert/.shader/.wgsl/.css/.scss/.sass/.less/.svg/.min.*/.map`) + 8 new line-suppression rules (GLSL keywords, CSS-in-JS, inline SVG, ffmpeg `filter_complex`, User-Agent strings, SQL DDL, `throw new Error(\`...\`)`, markdown image URLs). Configurable via `.llm-security/policy.json` `entropy` section (thresholds, `suppress_extensions`, `suppress_line_patterns`, `suppress_paths`). Envelope `calibration` block reports skip counters + effective thresholds + policy source.
|
||||
3. **DEP typosquat allowlist expansion** — 22 npm + 5 PyPI entries for short-name tools that tripped Levenshtein detection on every modern codebase (`knip`, `oxlint`, `tsx`, `nx`, `rimraf`, `uv`, `ruff`, etc.).
|
||||
|
||||
See `docs/security-hardening-guide.md` §6 for the calibration story.
|
||||
|
|
|
|||
|
|
@ -824,7 +824,7 @@ This plugin provides full-stack security hardening (static analysis + supply cha
|
|||
|
||||
| Version | Date | Highlights |
|
||||
|---------|------|------------|
|
||||
| **7.0.0** | 2026-04-19 | **Trustworthy scoring (BREAKING).** Three changes target the false-positive cascade on real codebases (scan of hyperframes.com gave `BLOCK / Extreme / 100` with ~70% noise). **1. Risk-score v2** (`scanners/lib/severity.mjs`) — severity-dominated, log-scaled within tier. Replaces sum-and-cap that collapsed every non-trivial scan to 100/Extreme. Tiers: critical → 70–95, high only → 40–65, medium only → 15–35, low only → 1–11. Verdict cutoffs realigned (BLOCK ≥65, WARNING ≥15) for band co-monotonicity. **2. Context-aware entropy scanner** — file-extension skip (`.glsl/.frag/.vert/.shader/.wgsl/.css/.scss/.sass/.less/.svg/.min.*/.map`) + 7 new line-suppression rules (GLSL keywords, CSS-in-JS templates, inline SVG, ffmpeg `filter_complex`, User-Agent strings, SQL DDL on dedicated lines, `throw new Error(\`...\`)`). Configurable via `.llm-security/policy.json` `entropy` section (thresholds, `suppress_extensions`, `suppress_line_patterns`, `suppress_paths`). Envelope `calibration` block reports skip counters + effective thresholds + policy source. **3. DEP typosquat allowlist expansion** — 22 npm + 5 PyPI entries for short-name tools that tripped Levenshtein on every modern codebase (`knip`, `oxlint`, `tsx`, `nx`, `rimraf`, `uv`, `ruff`, etc.). Synthesizer "Scan Calibration" section + "never override verdict" rule added. Legacy `riskScoreV1()` kept for reference. **CI pipelines with `--fail-on` thresholds may need recalibration.** 1485 tests (was 1461). |
|
||||
| **7.0.0** | 2026-04-19 | **Trustworthy scoring (BREAKING).** Three changes target the false-positive cascade on real codebases (scan of hyperframes.com gave `BLOCK / Extreme / 100` with ~70% noise). **1. Risk-score v2** (`scanners/lib/severity.mjs`) — severity-dominated, log-scaled within tier. Replaces sum-and-cap that collapsed every non-trivial scan to 100/Extreme. Tiers: critical → 70–95, high only → 40–65, medium only → 15–35, low only → 1–11. Verdict cutoffs realigned (BLOCK ≥65, WARNING ≥15) for band co-monotonicity. **2. Context-aware entropy scanner** — file-extension skip (`.glsl/.frag/.vert/.shader/.wgsl/.css/.scss/.sass/.less/.svg/.min.*/.map`) + 8 new line-suppression rules (GLSL keywords, CSS-in-JS templates, inline SVG, ffmpeg `filter_complex`, User-Agent strings, SQL DDL on dedicated lines, `throw new Error(\`...\`)`, markdown image URLs). Configurable via `.llm-security/policy.json` `entropy` section (thresholds, `suppress_extensions`, `suppress_line_patterns`, `suppress_paths`). Envelope `calibration` block reports skip counters + effective thresholds + policy source. **3. DEP typosquat allowlist expansion** — 22 npm + 5 PyPI entries for short-name tools that tripped Levenshtein on every modern codebase (`knip`, `oxlint`, `tsx`, `nx`, `rimraf`, `uv`, `ruff`, etc.). Synthesizer "Scan Calibration" section + "never override verdict" rule added. Legacy `riskScoreV1()` kept for reference. **CI pipelines with `--fail-on` thresholds may need recalibration.** 1487 tests (was 1461). |
|
||||
| **6.6.0** | 2026-04-18 | **JetBrains/IntelliJ plugin scanning.** `/security ide-scan` now covers JetBrains IDEs (IntelliJ IDEA, PyCharm, GoLand, WebStorm, RubyMine, PhpStorm, CLion, DataGrip, RustRover, Rider, Aqua, Writerside, Android Studio) — Fleet and Toolbox excluded. OS-aware discovery of `~/Library/Application Support/JetBrains/<IDE><version>/plugins/` (macOS), `%APPDATA%\JetBrains\...` (Windows), `~/.config/JetBrains/...` (Linux). Zero-dep parsers for `META-INF/plugin.xml` and `META-INF/MANIFEST.MF` with nested-jar extraction. 7 JetBrains-specific checks: theme-with-code, broad activation (`application-components`), `Premain-Class` instrumentation (HIGH — javaagent retransform), native binaries (`.so`/`.dylib`/`.dll`/`.jnilib`), long `<depends>` chains (supply-chain pressure), typosquat vs top JetBrains plugins, shaded-jar advisory. URL fetch for `plugins.jetbrains.com/plugin/<numericId>-<slug>` + direct `/plugin/download?pluginId=<xmlId>`; metadata resolves numericId → xmlId before download. `.kt`, `.groovy`, `.scala` added to `taint-tracer` code extensions. Reuses existing OS sandbox (`lib/vsix-sandbox.mjs` parameterized via `buildSandboxedWorker(..., workerPath)`). Knowledge: `knowledge/jetbrains-marketplace-api-notes.md`, expanded `knowledge/ide-extension-threat-patterns.md`, seeded `knowledge/top-jetbrains-plugins.json`. 1461 tests (was 1352). |
|
||||
| **6.5.0** | 2026-04-17 | **OS sandbox for `/security ide-scan <url>`.** VSIX fetch + extract now runs in a sub-process wrapped by `sandbox-exec` (macOS) or `bwrap` (Linux), reusing the same primitives proven by the v5.1 git-clone sandbox. Defense-in-depth — even if `lib/zip-extract.mjs` ever has a bypass, the kernel refuses any write outside the per-scan temp directory. New: `lib/vsix-fetch-worker.mjs` (sub-process worker with deterministic JSON-line IPC) and `lib/vsix-sandbox.mjs` (`buildSandboxProfile` / `buildBwrapArgs` / `buildSandboxedWorker` / `runVsixWorker`, 35 s timeout, 1 MB stdout cap). New `scan(target, { useSandbox })` option (default `true` for CLI; tests use `false` since `globalThis.fetch` mocks do not cross processes). Windows fallback: in-process with `meta.warnings` advisory. Envelope `meta.source.sandbox` field: `'sandbox-exec' \| 'bwrap' \| 'none' \| 'in-process'`. 1352 tests (was 1344). |
|
||||
| **6.4.0** | 2026-04-17 | **`/security ide-scan <url>` — pre-install verification.** The IDE extension scanner now accepts URLs and fetches the VSIX before scanning. Supported: VS Code Marketplace (`https://marketplace.visualstudio.com/items?itemName=publisher.name`), OpenVSX (`https://open-vsx.org/extension/publisher/name[/version]`), and direct `.vsix` URLs. New libraries: `lib/vsix-fetch.mjs` (HTTPS-only fetch with 50MB cap, 30s timeout, SHA-256, manual host-whitelisted redirects) and `lib/zip-extract.mjs` (zero-dep ZIP parser, rejects zip-slip / symlinks / absolute paths / drive letters / encrypted entries / ZIP64; caps: 10 000 entries, 500MB uncompressed, 100x expansion ratio, depth 20). Temp dir always cleaned in `try/finally`. Envelope `meta.source` carries `{ type: "url", kind, url, finalUrl, sha256, size, publisher, name, version }`. New knowledge file: `marketplace-api-notes.md`. GitHub repo URLs intentionally not supported (would require a build step). 1344 tests (was 1296). |
|
||||
|
|
|
|||
|
|
@ -192,11 +192,13 @@ suppression layers:
|
|||
.css, .scss, .sass, .less, .svg` + compound `.min.js, .min.css, .map`. A
|
||||
skip counter (`calibration.files_skipped_by_extension`) is reported in the
|
||||
scanner envelope.
|
||||
2. **Line-level rules 11–17** — applied when a line contains any of: GLSL
|
||||
2. **Line-level rules 11–18** — applied when a line contains any of: GLSL
|
||||
keywords (`uniform`, `vec3`, `texture2D`…), CSS-in-JS templates
|
||||
(`styled.…`), inline `<svg>` markup, ffmpeg `filter_complex` syntax,
|
||||
browser `User-Agent` strings, SQL DDL on a dedicated line
|
||||
(`^\s*(SELECT|INSERT|…)`), or `throw new Error(\`…\`)` templates.
|
||||
(`^\s*(SELECT|INSERT|…)`), `throw new Error(\`…\`)` templates, or
|
||||
markdown image syntax with external URL (`` — common
|
||||
in JSON content indexes / article metadata).
|
||||
3. **Per-project policy override** — `.llm-security/policy.json` `entropy`
|
||||
section supports:
|
||||
|
||||
|
|
|
|||
|
|
@ -169,6 +169,13 @@ const SQL_STATEMENT = /^\s*(?:SELECT|INSERT|UPDATE|DELETE|CREATE|ALTER|WITH|DROP
|
|||
/** Error-message templates with embedded HTML/markup (throw new Error("<div>...</div>")). */
|
||||
const ERROR_TEMPLATE = /(?:throw\s+new\s+(?:Error|TypeError|RangeError|SyntaxError)|new\s+Error\s*\()\s*[`'"]/;
|
||||
|
||||
/**
|
||||
* Markdown image syntax with external URL — ``.
|
||||
* Common in JSON data indexes / article metadata; CDN URL hash segments
|
||||
* produce high Shannon entropy but are not credentials.
|
||||
*/
|
||||
const MARKDOWN_IMAGE = /!\[[^\]]*\]\(\s*https?:\/\//;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// False-positive suppression helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
|
@ -248,7 +255,10 @@ function isFalsePositive(str, line, absPath) {
|
|||
// 17. Error-message templates (throw new Error("<html>...</html>"))
|
||||
if (ERROR_TEMPLATE.test(line)) return true;
|
||||
|
||||
// 18. User-policy regex patterns from .llm-security/policy.json
|
||||
// 18. Markdown image syntax with external URL — CDN hash noise in content repos
|
||||
if (MARKDOWN_IMAGE.test(line)) return true;
|
||||
|
||||
// 19. User-policy regex patterns from .llm-security/policy.json
|
||||
for (const pattern of USER_SUPPRESS_LINE_PATTERNS) {
|
||||
if (pattern.test(line)) return true;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -186,6 +186,28 @@ describe('entropy-scanner context suppression (v7.0.0+)', () => {
|
|||
assert.equal(result.findings.length, 0, 'expected throw new Error line to suppress');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rule 18: markdown image with external URL suppresses finding', async () => {
|
||||
const fx = await newRoot('ent-rule18-');
|
||||
await writeFixture(fx, 'index.json',
|
||||
'{"summary": ""}');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.equal(result.findings.length, 0, 'expected markdown image line to suppress');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rule 18 does NOT over-match plain URLs without image syntax', async () => {
|
||||
const fx = await newRoot('ent-rule18b-');
|
||||
await writeFixture(fx, 'app.js',
|
||||
'const token = "' + PAYLOAD + '"; // not an image');
|
||||
resetCounter();
|
||||
const discovery = await discoverFiles(fx);
|
||||
const result = await scan(fx, discovery);
|
||||
assert.ok(result.findings.length >= 1, 'plain high-entropy string must still be detected');
|
||||
await rm(fx, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('C. Policy-driven overrides', () => {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue