open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity

test(llm-security): build poisoned fixtures at test time, never on disk

Browse source 
v8.1.0 AV surface, session S1. The three poisoned fixture trees
(signature-scan/poisoned, memory-scan/poisoned-project, trigger-scan/poisoned)
are deleted from disk and materialized into a temp dir by the new
tests/helpers/payload-trees.mjs. SIG-matching strings are assembled from
fragments, the zero-width carrier comes from String.fromCodePoint, and every
file carries the sha256 of the retired on-disk bytes;
tests/helpers/payload-trees.test.mjs asserts the materialized trees are
byte-identical (mutation-checked: one changed byte fails it).

Inline payload literals in signature-scanner, signature-scanner-custom-rules
and e2e/scan-pipeline are fragmented the same way; the literal U+200B in
attack-simulator, auto-cleaner-rce and auto-cleaner-traversal is replaced by
String.fromCodePoint(0x200B).

av-surface: a 3->0, a2 3->0, c 5->1, d 5->2, b 9->8 (webshell-b64 blob gone).
What remains (c=1, d=2, b) is under examples/** or is (b), both S2.
Suite 2261 / 2252 pass / 3 fail (av-surface b, c, d only) / 6 skip.
Golden output identical before/after (109/7/4, 61/61).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-22 13:07:38 +02:00
commit 31aa2b4943
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
21 changed files with 282 additions and 143 deletions
Download patch file Download diff file Expand all files Collapse all files

10
tests/scanners/memory-poisoning.test.mjs
View file

@ -1,20 +1,24 @@
// memory-poisoning.test.mjs — Integration tests for the memory-poisoning-scanner
// Tests against fixtures in tests/fixtures/memory-scan/ with:
// - clean-project: normal CLAUDE.md + memory file + rules (0 findings expected)
// - poisoned-project: injection, shell commands, credential paths, suspicious URLs,
// - poisoned-project (materialized at test time by
// tests/helpers/payload-trees.mjs, never on disk): injection, shell commands, credential paths, suspicious URLs,
// permission expansion, encoded payloads
import { describe, it, beforeEach } from 'node:test';
import { describe, it, beforeEach, after } from 'node:test';
import assert from 'node:assert/strict';
import { resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { resetCounter } from '../../scanners/lib/output.mjs';
import { discoverFiles } from '../../scanners/lib/file-discovery.mjs';
import { scan } from '../../scanners/memory-poisoning-scanner.mjs';
import { materializeTree } from '../helpers/payload-trees.mjs';
const __dirname = fileURLToPath(new URL('.', import.meta.url));
const CLEAN_FIXTURE = resolve(__dirname, '../fixtures/memory-scan/clean-project');
const POISONED_FIXTURE = resolve(__dirname, '../fixtures/memory-scan/poisoned-project');
const poisoned = materializeTree('memory-scan/poisoned-project');
after(poisoned.cleanup);
const POISONED_FIXTURE = poisoned.dir;
// ---------------------------------------------------------------------------
// Clean project — should produce 0 findings