llm-security/tests/golden
Kjell Tore Guttormsen 90f576f056 fix(llm-security): v8.x-A - close <script> recall hole, add whole-table ReDoS gate
Two of the three confirmed v8.x-A evasions are closed; the third (attribute
padding) stays open by operator decision and is documented, not silently left.

Recall fix. hybrid-xss required a closing </script>, so `<script>alert(1)` and
`<script src=x.js>` both passed scanForInjection() with found: false, while the
closed form returned high. A src= tag has no body to close in the first place.
The opening tag alone is the signal, and matching it is strictly linear: one
negated-class run whose excluded character is its own terminator, so there is
no backtracking surface that could re-introduce v7.8.3 #24.

ReDoS gate. The #24 test covered six html-obfuscation patterns against the two
shapes that defect was found on. A catastrophically backtracking regex added
anywhere else in the four tables would have failed no test at all - which is
why #24 had to be found by hand. The new gate times every exported pattern and
asserts its own coverage, so it cannot be narrowed silently.

It has two classes because the blowup shapes need opposite inputs, and because
a synchronous RegExp.test() cannot be interrupted: an exponential pattern met
with a 512KB input would HANG the run rather than fail it. Class 1 uses a
28-char ambiguous-run ladder, where exponential costs ~1s and anything sane
costs microseconds; on a hit the 512KB sweeps refuse to run. Class 2 uses the
hook's real read cap to catch the polynomial #24 class. Proven to fire: an
injected exponential regex was named by label and the hang guard held.

Measured clean: worst single pattern 17ms, full large sweep 218ms.

Golden gate went red as expected and was re-blessed - diff is exactly two
lines, the regex source and the source-file digest. The 61-payload reference
run is byte-identical, so no showcase behaviour moved.

Suite 2073/2073 (was 2063; +5 recall tests, +5 gate tests).

Still open, unchanged: all 7 bounded HTML patterns evade on >256 chars of
padding, in two positions - before the attribute and inside the style value.
Wider than STATE recorded, which named only aria-label.
2026-08-10 14:53:29 +02:00
..
patterns.json fix(llm-security): v8.x-A - close <script> recall hole, add whole-table ReDoS gate 2026-08-10 14:53:29 +02:00
README.md fix(llm-security): golden gate - coverage block measured something else 2026-08-09 13:06:45 +02:00
reference-run.json fix(llm-security): golden gate - coverage block measured something else 2026-08-09 13:06:45 +02:00
suite-counts.json test(llm-security): v8 Phase 5 step 1 - golden baseline before any table swap 2026-08-09 12:57:51 +02:00

Golden baseline — v8 Phase 5

Reference artifacts recorded before any commons extraction, so that each table-by-table swap in Phase 5 can be proven behaviour-preserving (or rolled back). Generated and checked by one code path:

node scripts/golden-baseline.mjs            # check only, exits 1 on drift
node scripts/golden-baseline.mjs --write    # (re-)bless
node scripts/golden-baseline.mjs --write --suite   # also refresh suite counts (slow)

The gate is tests/lib/golden-baseline.test.mjs.

The artifacts

File What it pins Why that layer exists
patterns.json .source + .flags of every RegExp reachable from the walked modules' exports After a swap a pattern is new RegExp(jsonString, flags). The plan's named hazard — JSON backslash-doubling — is visible only on the compiled object.
↳ table records key/value digest of HOMOGLYPH_MAP, TYPOSQUAT_SUSPICIOUS_TOKENS, SEVERITY, the four OWASP maps Most of what Phase 4 moves is not a regex. A regex-only dump is blind to a broken homoglyph or OWASP-map swap, i.e. blind to the bulk of the payload.
↳ file records sha256 of the five source files in the moving set The completeness layer, complete by construction: it covers regexes inlined in function bodies (NAMED at string-utils.mjs:291, the BIDI/tag/PUA ranges at 357404) that no export walk can reach.
reference-run.json the 61 showcase payloads through the real hook entry points, plus a static reachability probe Byte identity over a corpus that trips a handful of patterns would prove almost nothing, so reachability is recorded and unreachable patterns are listed by name.
suite-counts.json per-file pass/fail, each file run alone A total is unattributable, and npm test runs files concurrently where three timing-sensitive files flake. Running each alone is the only reproducible recording.

Why there is no regex enumerator

The obvious design — parse the sources and enumerate every regex literal — was rejected. It needs a JS parser this zero-dependency repo does not have, and a lexical approximation is contaminated by comments and division (severity.mjs scores 4 "regexes" that way and exports none). The two-layer split — export walk for what the scanners actually use, file digest for everything else — answers the same question without a parser.

During a swap

A diff here means the swap changed observable behaviour. Roll the swap back. --write is for deliberately re-blessing a change you have already decided is correct, not for making the gate quiet.

Known coverage gaps — read before quoting a number

coverage is static reachability, NOT observed coverage. It probes the 61 payload strings against the 83 injection patterns in-process: "if you threw every payload string at every regex, how many would match?" It does not measure what the 61 hook invocations evaluated — the pre-bash-destructive payloads never reach injection-patterns at all, yet their strings are in the probe set and can mark a pattern reachable. 47/83 is therefore an upper bound on what the corpus could protect, not a measurement of what it did.

The block is named kind: "static-reachability" and carries that caveat in a note field; the gate asserts both, so the honest label cannot be dropped quietly.

Two gaps follow:

  • 36 unreachable patterns, listed by key under coverage.unreachablePatterns. A swap that breaks one of those is caught by the pattern dump only. Closing it means growing the conformance corpus (Phase 5 step 5).
  • The four OWASP maps have no behavioural coverage at all. They are scanner-side and no hook in this corpus reaches them. They were the reason the dump was widened to table records, and the digest is their only protection — do not read the reference run as backing them.

corpusContains says a payload contains a homoglyph or is altered by normalizeForScan. It does not say the run folded one.