open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity

test(llm-security): build poisoned fixtures at test time, never on disk

Browse source 
v8.1.0 AV surface, session S1. The three poisoned fixture trees
(signature-scan/poisoned, memory-scan/poisoned-project, trigger-scan/poisoned)
are deleted from disk and materialized into a temp dir by the new
tests/helpers/payload-trees.mjs. SIG-matching strings are assembled from
fragments, the zero-width carrier comes from String.fromCodePoint, and every
file carries the sha256 of the retired on-disk bytes;
tests/helpers/payload-trees.test.mjs asserts the materialized trees are
byte-identical (mutation-checked: one changed byte fails it).

Inline payload literals in signature-scanner, signature-scanner-custom-rules
and e2e/scan-pipeline are fragmented the same way; the literal U+200B in
attack-simulator, auto-cleaner-rce and auto-cleaner-traversal is replaced by
String.fromCodePoint(0x200B).

av-surface: a 3->0, a2 3->0, c 5->1, d 5->2, b 9->8 (webshell-b64 blob gone).
What remains (c=1, d=2, b) is under examples/** or is (b), both S2.
Suite 2261 / 2252 pass / 3 fail (av-surface b, c, d only) / 6 skip.
Golden output identical before/after (109/7/4, 61/61).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-22 13:07:38 +02:00
commit 31aa2b4943
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
21 changed files with 282 additions and 143 deletions
Download patch file Download diff file Expand all files Collapse all files

10
tests/e2e/scan-pipeline.test.mjs
View file

@ -15,7 +15,8 @@
// - The exit-code contract: 0 (PASS), 1 (WARNING), 2 (BLOCK).
//
// Two contrasting fixtures:
// POISONED: tests/fixtures/memory-scan/poisoned-project — multi-vector
// POISONED: the memory-scan/poisoned-project tree, materialized at test
// time by tests/helpers/payload-trees.mjs (never on disk) — multi-vector
// attack: tampered CLAUDE.md, suspicious git history, network leaks,
// embedded credentials, etc. Must produce BLOCK verdict.
// CLEAN: tests/fixtures/posture-scan/grade-a-project — well-built
@ -33,10 +34,13 @@ import { fileURLToPath } from 'node:url';
import { spawn, spawnSync } from 'node:child_process';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { materializeTree } from '../helpers/payload-trees.mjs';
const __dirname = dirname(fileURLToPath(import.meta.url));
const ORCHESTRATOR = resolve(__dirname, '../../scanners/scan-orchestrator.mjs');
const POISONED = resolve(__dirname, '../fixtures/memory-scan/poisoned-project');
const poisonedTree = materializeTree('memory-scan/poisoned-project');
after(poisonedTree.cleanup);
const POISONED = poisonedTree.dir;
const CLEAN = resolve(__dirname, '../fixtures/posture-scan/grade-a-project');
// The v7.8.0 deep-scan scanners (trg/sig/ast) are orchestrated alongside the
@ -282,7 +286,7 @@ describe('e2e scan-pipeline — v7.8.0 scanners surface through the aggregate',
// SIG: a classic PHP webshell (SIG-WEBSHELL-001, critical) — same shape as
// the signature-scan poisoned fixture.
writeFileSync(join(dir, 'evil.php'), "<?php @eval($_POST['x']); ?>\n");
writeFileSync(join(dir, 'evil.php'), "<?php @ev" + "al($_POST['x']); ?>\n");
// AST: os.environ (source) -> requests.post (sink) through an intermediate
// variable (AST-NET-EXFIL). Only asserted when python3 is present.