open/llm-security
1
0
Fork 0
Code Issues Releases 6 Activity

test(llm-security): narrow av-surface probe (b) to a command shape

Browse source 
The S0 rule fired on 'http' anywhere in decoded text and flagged an SVG
data URI (xmlns) in the hyperframes-like skill-scan fixture - markup, not
a command. PM decision for S2: the decoded text must start with a command
word or contain a pipe into a shell or $(. Measured before any blob was
removed: 8 blobs/7 files -> 8 blobs/6 files; SVG out, one split-payload
fragment in security-assessment.md newly caught, all 7 command blobs kept.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-22 13:17:02 +02:00
commit daa12b3bbb
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
1 changed files with 15 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

21
tests/av-surface.test.mjs
View file

@ -23,10 +23,19 @@
// NOT gated here; S1 moves them to test-time construction anyway.
// - scanners/commons/**, the vendored pull-only subtree this
// repository may not edit.
// (b) a base64 blob of 24+ characters that decodes to printable text
// containing curl, wget, bash, sh, eval or http as a WORD. Word
// boundaries are deliberate: a bare substring `sh` matches "should" and
// "hash", which are not shell commands.
// (b) a base64 blob of 24+ characters that decodes to printable text with
// a COMMAND SHAPE: it starts with curl, wget, bash, sh, python, eval or
// nc as a word, or it contains a pipe into bash/sh or `$(`. Word
// boundaries are deliberate: a bare substring `sh` matches "should".
// Narrowed in S2 (2026-09-22, PM decision): the S0 rule fired on `http`
// ANYWHERE in the decoded text, which caught an SVG data URI in
// tests/fixtures/skill-scan/hyperframes-like (xmlns="http://www.w3.org/
// 2000/svg") — markup, not a command, and not what an AV reacts to.
// Measured before the real blobs were removed: 8 blobs in 7 files under
// the S0 rule, 8 in 6 under this one — the SVG dropped out, and `$(`
// caught the second half of a payload split across two blobs in
// examples/malicious-skill-demo/security-assessment.md, which the S0
// rule missed. All 7 command blobs the S0 rule saw still hit.
// (c) a Unicode Tag (U+E0000-U+E007F), zero-width (U+200B-U+200D, U+2060,
// U+FEFF) or bidi-control (U+202A-U+202E, U+2066-U+2069) codepoint in a
// TEXT file outside scanners/commons/conformance/**. Binary files (a NUL
@ -68,7 +77,7 @@ const COMMENT_LINE = /^\s*(?:\/\/|#(?!!)|\/\*|\*|<!--)/;
const BASE64_BLOB = /[A-Za-z0-9+/]{24,}={0,2}/g;
const PRINTABLE = /^[\x20-\x7e\t\r\n]+$/;
const SHELL_WORD = /\b(?:curl|wget|bash|sh|eval|http)\b/;
const COMMAND_SHAPE = /^\s*(?:curl|wget|bash|sh|python[0-9.]*|eval|nc)\b|\|\s*(?:bash|sh)\b|\$\(/;
const CARRIER = /[\u{E0000}-\u{E007F}\u200B-\u200D\u2060\uFEFF\u202A-\u202E\u2066-\u2069]/u;
@ -122,7 +131,7 @@ function measureAvSurface() {
for (const m of text.matchAll(BASE64_BLOB)) {
const decoded = Buffer.from(m[0], 'base64').toString('latin1');
if (PRINTABLE.test(decoded) && SHELL_WORD.test(decoded)) {
if (PRINTABLE.test(decoded) && COMMAND_SHAPE.test(decoded)) {
b.push(`${rel} :: ${decoded.slice(0, 60).replace(/\s+/g, ' ')}`);
}
}