open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity

test(llm-security): build poisoned fixtures at test time, never on disk

Browse source 
v8.1.0 AV surface, session S1. The three poisoned fixture trees
(signature-scan/poisoned, memory-scan/poisoned-project, trigger-scan/poisoned)
are deleted from disk and materialized into a temp dir by the new
tests/helpers/payload-trees.mjs. SIG-matching strings are assembled from
fragments, the zero-width carrier comes from String.fromCodePoint, and every
file carries the sha256 of the retired on-disk bytes;
tests/helpers/payload-trees.test.mjs asserts the materialized trees are
byte-identical (mutation-checked: one changed byte fails it).

Inline payload literals in signature-scanner, signature-scanner-custom-rules
and e2e/scan-pipeline are fragmented the same way; the literal U+200B in
attack-simulator, auto-cleaner-rce and auto-cleaner-traversal is replaced by
String.fromCodePoint(0x200B).

av-surface: a 3->0, a2 3->0, c 5->1, d 5->2, b 9->8 (webshell-b64 blob gone).
What remains (c=1, d=2, b) is under examples/** or is (b), both S2.
Suite 2261 / 2252 pass / 3 fail (av-surface b, c, d only) / 6 skip.
Golden output identical before/after (109/7/4, 61/61).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-22 13:07:38 +02:00
commit 31aa2b4943
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
21 changed files with 282 additions and 143 deletions
Download patch file Download diff file Expand all files Collapse all files

18
tests/scanners/trigger-scanner.test.mjs
View file

@ -1,10 +1,13 @@
// trigger-scanner.test.mjs — Tests for the TRG trigger/activation-abuse scanner.
// Fixtures in tests/fixtures/trigger-scan/:
// - clean/ : a scoped, specifically-described skill (0 findings expected)
// - poisoned/ : a built-in-shadowing command (read), a broad+baiting skill (run),
// and an obfuscated-baiting agent (zero-width space inside "anything")
// Fixtures:
// - tests/fixtures/trigger-scan/clean/ : a scoped, specifically-described
// skill (0 findings expected)
// - poisoned (materialized at test time by tests/helpers/payload-trees.mjs,
// never on disk): a built-in-shadowing command (read), a broad+baiting
// skill (run), and an obfuscated-baiting agent (zero-width space inside
// "anything")
import { describe, it, beforeEach } from 'node:test';
import { describe, it, beforeEach, after } from 'node:test';
import assert from 'node:assert/strict';
import { resolve, join } from 'node:path';
import { fileURLToPath } from 'node:url';
@ -13,10 +16,13 @@ import { tmpdir } from 'node:os';
import { resetCounter } from '../../scanners/lib/output.mjs';
import { discoverFiles } from '../../scanners/lib/file-discovery.mjs';
import { scan } from '../../scanners/trigger-scanner.mjs';
import { materializeTree } from '../helpers/payload-trees.mjs';
const __dirname = fileURLToPath(new URL('.', import.meta.url));
const CLEAN_FIXTURE = resolve(__dirname, '../fixtures/trigger-scan/clean');
const POISONED_FIXTURE = resolve(__dirname, '../fixtures/trigger-scan/poisoned');
const poisoned = materializeTree('trigger-scan/poisoned');
after(poisoned.cleanup);
const POISONED_FIXTURE = poisoned.dir;
// ---------------------------------------------------------------------------
// Clean — scoped skill, no trigger abuse