fix(llm-security): the ReDoS gate timed every pattern and reached 8 of 45

The v8.x-A whole-table gate times every exported pattern against a corpus
of 8 hand-written units and asserts its own coverage -- but the assertion
`covers every exported pattern` guards the pattern LIST, not the input
corpus. A pattern is only measured if some unit happens to carry its
leading literal; otherwise it fails on the first character and reports
green having measured nothing.

Measured on the pre-swap tables: 37 of the 45 prefix-bearing patterns
were never reached, including BOTH quadratic hybrid-xss rows this gate
was believed to cover. `<script ` and `<iframe ` appear in no unit, so
the two rows commons independently measured as quadratic ran their
literal-prefix check and stopped. Same defect class as all of v7.8.2:
reported success without running.

Hand-writing 37 more units does not fix it -- it re-arms the same trap at
the next pattern. Class 3 derives each attack unit from the pattern's OWN
literal prefix, so coverage is a function of the table rather than a list
someone must remember to extend. 64KB rather than the 512KB read cap for
the class-1 reason: a quadratic pattern met at 512KB stalls the run for
minutes instead of failing it.

Proven to fire, both directions, against the vendored file:
  - gate written first, pre-swap: RED, naming script-tag 1429ms and
    iframe-src 1161ms against a 150ms budget (exit 1)
  - post-swap: GREEN, 17.7ms for all 45 probes (exit 0)
  - vendored JSON mutated back to [^>]*: golden AND ReDoS gates both exit 1
  - vendored JSON corrupted: golden exit 1, conformance 3 fail
  - restored: all green

Clean-table margin at 64KB is ~700x: worst legitimate pattern 1.66ms.

Carried with the commons v0.4.3 subtree pull, which is what makes the
gate passable. v0.4.0 was the tag commons announced; v0.4.1-v0.4.3 came
after and touch no data table -- lexicon 0.8.0 and secret-egress
0.3.0/19 are identical across all four -- so v0.4.3 was taken for the
conformance manifest correction (302625e) they sent separately.

Golden re-blessed after a post-by-post diff: exactly 2 changed records,
both [^>]* -> [^><]*, 0 added, 0 removed, reference run 61/61 unchanged.
The file-sha256 layer did NOT move, contrary to the note in STATE: it
pins scanners/lib/injection-patterns.mjs, which has held no literals
since be14867. The vendored lexicon is covered by the regex layer only.

The script-tag tripwire pinned the old form and fired correctly. Updated
to the v0.4.x form and widened to the iframe row, which had no tripwire
while it was quadratic -- which is why nobody had named it.

Full suite 2193 pass / 6 skipped. The one red is the documented
pre-compact size-cap timing flake; passes alone (exit 0), as do
attack-simulator and the gate itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJxU3xuwfMq8W1mxtiGhLk
This commit is contained in:
Kjell Tore Guttormsen 2026-08-13 20:30:31 +02:00
commit 18bc1dc92e
3 changed files with 101 additions and 8 deletions

View file

@ -352,7 +352,7 @@
{
"kind": "regex",
"key": "injection-patterns:HYBRID_PATTERNS[4].pattern",
"source": "<script\\b[^>]*>",
"source": "<script\\b[^><]*>",
"flags": "i"
},
{
@ -370,7 +370,7 @@
{
"kind": "regex",
"key": "injection-patterns:HYBRID_PATTERNS[7].pattern",
"source": "<iframe\\b[^>]*src\\s*=\\s*[\"'][^\"']*(?:javascript:|data:text\\/html)",
"source": "<iframe\\b[^><]*src\\s*=\\s*[\"'][^\"']*(?:javascript:|data:text\\/html)",
"flags": "i"
},
{

View file

@ -87,14 +87,34 @@ describe('injection-lexicon (commons lexicon)', () => {
]);
});
it('carries commons v0.3.0\'s converged script-tag form', () => {
// The one detection value that changed in v0.2.0..v0.3.0. Named here so
// a future commons that re-adds the `[\s\S]*?<\/script>` tail — the
// recall hole we asked them to drop — fails loudly rather than silently
// narrowing what we detect.
it('carries commons v0.4.x\'s linear script-tag and iframe-src forms', () => {
// These two rows are the detection values that have moved under us, and
// they now carry TWO properties worth pinning, not one:
//
// recall — no `[\s\S]*?<\/script>` tail. That tail was the recall hole
// we asked commons to drop in v0.3.0, and re-adding it would silently
// narrow what we detect.
//
// cost — the negated class excludes `<` as well as `>`. Under the old
// `[^>]*` both rows were quadratic: measured 2026-08-13 on 64KB of
// their own prefix, script-tag 1429ms and iframe-src 1161ms, growing
// 4x per doubling. At the hook read caps that is a ~31s stall in
// pre-compact and ~2min in a remote `/security scan <url>`, on
// attacker-controlled input. `[^><]*` is linear and has no bound to pad
// past, so it beats `{0,256}` on both axes rather than trading them.
//
// Both rows are named because both moved. The iframe row had no tripwire
// when it was quadratic, which is why nobody had named it.
const scriptTag = HYBRID_PATTERNS.find((p) => p.label.startsWith('hybrid-xss: <script>'));
assert.equal(scriptTag.pattern.source, '<script\\b[^>]*>');
assert.equal(scriptTag.pattern.source, '<script\\b[^><]*>');
assert.equal(scriptTag.pattern.flags, 'i');
const iframeSrc = HYBRID_PATTERNS.find((p) => p.label.startsWith('hybrid-xss: iframe'));
assert.equal(
iframeSrc.pattern.source,
'<iframe\\b[^><]*src\\s*=\\s*["\'][^"\']*(?:javascript:|data:text\\/html)',
);
assert.equal(iframeSrc.pattern.flags, 'i');
});
});

View file

@ -1373,6 +1373,79 @@ describe('injection patterns — whole-table ReDoS gate (v8.x-A)', () => {
`full-table sweep took ${ms.toFixed(0)}ms, budget ${TOTAL_BUDGET_MS}ms`);
});
// Class 3 — the corpus above is hand-written, and a hand-written corpus is
// exactly what does not survive the next pattern. Both classes reach a
// pattern only if some unit happens to contain its leading literal, and
// `covers every exported pattern` guards the pattern LIST, not the input
// corpus — so a pattern whose prefix no unit carries is timed against input
// it rejects on the first character, and reports green having measured
// nothing. Measured 2026-08-13 on the pre-swap tables: 37 of the 45
// prefix-bearing patterns were never reached, including both quadratic
// hybrid-xss rows this gate was believed to cover.
//
// The fix is to stop hand-writing the corpus. Each pattern gets an attack
// unit built from its OWN literal prefix, so coverage is a function of the
// table rather than a list someone has to remember to extend.
//
// 64KB rather than the 512KB read cap, for the class-1 reason: a quadratic
// pattern met at 512KB stalls the run for minutes instead of failing it.
// 64KB already separates the two classes by ~700x — measured on the pre-swap
// tables, worst clean pattern 1.66ms against iframe-src 1641ms and
// script-tag 1184ms, both growing 4x per doubling.
const SELF_PREFIX_CAP = 64 * 1024;
const SELF_PREFIX_BUDGET_MS = 150;
// Leading run of plain literals, stopping at the first metacharacter or at a
// literal that a following quantifier makes optional.
const REGEX_META = new Set([...'\\^$.|?*+()[]{}']);
const literalPrefix = (source) => {
let out = '';
for (let i = 0; i < source.length; i += 1) {
if (REGEX_META.has(source[i])) break;
const next = source[i + 1];
if (next === '?' || next === '*' || next === '{') break;
out += source[i];
}
return out;
};
const selfPrefixProbes = ALL_PATTERNS
.map((entry) => ({ ...entry, prefix: literalPrefix(entry.pattern.source) }))
// A prefix under 2 chars is not a cheap-fail anchor: those patterns start
// on a class or group and both classes above already engage them.
.filter(({ prefix }) => prefix.length >= 2);
it('probes every pattern that has a literal prefix to fail cheaply on', () => {
// Guards this class the way `covers every exported pattern` guards the
// list — but on the axis that was actually blind. If the extractor stops
// recognising prefixes, this count collapses and says so.
assert.ok(selfPrefixProbes.length >= 40,
`expected the prefix-bearing patterns, got ${selfPrefixProbes.length} of ${ALL_PATTERNS.length}`);
for (const { pattern, label, prefix } of selfPrefixProbes) {
const unit = grow(prefix, SELF_PREFIX_CAP);
assert.ok(unit.includes(prefix),
`probe input for ${label} does not contain its own prefix ${JSON.stringify(prefix)}`);
assert.equal(unit.length, SELF_PREFIX_CAP,
`probe input for ${label} is ${unit.length} chars, expected ${SELF_PREFIX_CAP}`);
assert.ok(pattern.source.startsWith(prefix),
`extracted prefix ${JSON.stringify(prefix)} is not a prefix of ${label}`);
}
});
it('no pattern backtracks polynomially on input built from its own prefix', () => {
assert.deepEqual(exponentialOffenders, [],
'refusing to run large input against an exponential pattern — fix the class-1 failure first');
const offenders = [];
for (const { pattern, label, prefix } of selfPrefixProbes) {
const ms = timeMs(pattern, grow(prefix, SELF_PREFIX_CAP));
if (ms > SELF_PREFIX_BUDGET_MS) {
offenders.push(`${label} :: prefix ${JSON.stringify(prefix)} :: ${ms.toFixed(0)}ms`);
}
}
assert.deepEqual(offenders, [],
`patterns exceeded ${SELF_PREFIX_BUDGET_MS}ms on ${SELF_PREFIX_CAP / 1024}KB of their own prefix:\n ${offenders.join('\n ')}`);
});
it('scanForInjection itself terminates on the pathological corpus', () => {
assert.deepEqual(exponentialOffenders, [],
'refusing to run 512KB input against an exponential pattern — fix the class-1 failure first');