fix(llm-security): the ReDoS gate timed every pattern and reached 8 of 45
The v8.x-A whole-table gate times every exported pattern against a corpus
of 8 hand-written units and asserts its own coverage -- but the assertion
`covers every exported pattern` guards the pattern LIST, not the input
corpus. A pattern is only measured if some unit happens to carry its
leading literal; otherwise it fails on the first character and reports
green having measured nothing.
Measured on the pre-swap tables: 37 of the 45 prefix-bearing patterns
were never reached, including BOTH quadratic hybrid-xss rows this gate
was believed to cover. `<script ` and `<iframe ` appear in no unit, so
the two rows commons independently measured as quadratic ran their
literal-prefix check and stopped. Same defect class as all of v7.8.2:
reported success without running.
Hand-writing 37 more units does not fix it -- it re-arms the same trap at
the next pattern. Class 3 derives each attack unit from the pattern's OWN
literal prefix, so coverage is a function of the table rather than a list
someone must remember to extend. 64KB rather than the 512KB read cap for
the class-1 reason: a quadratic pattern met at 512KB stalls the run for
minutes instead of failing it.
Proven to fire, both directions, against the vendored file:
- gate written first, pre-swap: RED, naming script-tag 1429ms and
iframe-src 1161ms against a 150ms budget (exit 1)
- post-swap: GREEN, 17.7ms for all 45 probes (exit 0)
- vendored JSON mutated back to [^>]*: golden AND ReDoS gates both exit 1
- vendored JSON corrupted: golden exit 1, conformance 3 fail
- restored: all green
Clean-table margin at 64KB is ~700x: worst legitimate pattern 1.66ms.
Carried with the commons v0.4.3 subtree pull, which is what makes the
gate passable. v0.4.0 was the tag commons announced; v0.4.1-v0.4.3 came
after and touch no data table -- lexicon 0.8.0 and secret-egress
0.3.0/19 are identical across all four -- so v0.4.3 was taken for the
conformance manifest correction (302625e) they sent separately.
Golden re-blessed after a post-by-post diff: exactly 2 changed records,
both [^>]* -> [^><]*, 0 added, 0 removed, reference run 61/61 unchanged.
The file-sha256 layer did NOT move, contrary to the note in STATE: it
pins scanners/lib/injection-patterns.mjs, which has held no literals
since be14867. The vendored lexicon is covered by the regex layer only.
The script-tag tripwire pinned the old form and fired correctly. Updated
to the v0.4.x form and widened to the iframe row, which had no tripwire
while it was quadratic -- which is why nobody had named it.
Full suite 2193 pass / 6 skipped. The one red is the documented
pre-compact size-cap timing flake; passes alone (exit 0), as do
attack-simulator and the gate itself.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJxU3xuwfMq8W1mxtiGhLk
This commit is contained in:
parent
464825f9bf
commit
18bc1dc92e
3 changed files with 101 additions and 8 deletions
|
|
@ -352,7 +352,7 @@
|
||||||
{
|
{
|
||||||
"kind": "regex",
|
"kind": "regex",
|
||||||
"key": "injection-patterns:HYBRID_PATTERNS[4].pattern",
|
"key": "injection-patterns:HYBRID_PATTERNS[4].pattern",
|
||||||
"source": "<script\\b[^>]*>",
|
"source": "<script\\b[^><]*>",
|
||||||
"flags": "i"
|
"flags": "i"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|
@ -370,7 +370,7 @@
|
||||||
{
|
{
|
||||||
"kind": "regex",
|
"kind": "regex",
|
||||||
"key": "injection-patterns:HYBRID_PATTERNS[7].pattern",
|
"key": "injection-patterns:HYBRID_PATTERNS[7].pattern",
|
||||||
"source": "<iframe\\b[^>]*src\\s*=\\s*[\"'][^\"']*(?:javascript:|data:text\\/html)",
|
"source": "<iframe\\b[^><]*src\\s*=\\s*[\"'][^\"']*(?:javascript:|data:text\\/html)",
|
||||||
"flags": "i"
|
"flags": "i"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|
|
||||||
|
|
@ -87,14 +87,34 @@ describe('injection-lexicon (commons lexicon)', () => {
|
||||||
]);
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('carries commons v0.3.0\'s converged script-tag form', () => {
|
it('carries commons v0.4.x\'s linear script-tag and iframe-src forms', () => {
|
||||||
// The one detection value that changed in v0.2.0..v0.3.0. Named here so
|
// These two rows are the detection values that have moved under us, and
|
||||||
// a future commons that re-adds the `[\s\S]*?<\/script>` tail — the
|
// they now carry TWO properties worth pinning, not one:
|
||||||
// recall hole we asked them to drop — fails loudly rather than silently
|
//
|
||||||
// narrowing what we detect.
|
// recall — no `[\s\S]*?<\/script>` tail. That tail was the recall hole
|
||||||
|
// we asked commons to drop in v0.3.0, and re-adding it would silently
|
||||||
|
// narrow what we detect.
|
||||||
|
//
|
||||||
|
// cost — the negated class excludes `<` as well as `>`. Under the old
|
||||||
|
// `[^>]*` both rows were quadratic: measured 2026-08-13 on 64KB of
|
||||||
|
// their own prefix, script-tag 1429ms and iframe-src 1161ms, growing
|
||||||
|
// 4x per doubling. At the hook read caps that is a ~31s stall in
|
||||||
|
// pre-compact and ~2min in a remote `/security scan <url>`, on
|
||||||
|
// attacker-controlled input. `[^><]*` is linear and has no bound to pad
|
||||||
|
// past, so it beats `{0,256}` on both axes rather than trading them.
|
||||||
|
//
|
||||||
|
// Both rows are named because both moved. The iframe row had no tripwire
|
||||||
|
// when it was quadratic, which is why nobody had named it.
|
||||||
const scriptTag = HYBRID_PATTERNS.find((p) => p.label.startsWith('hybrid-xss: <script>'));
|
const scriptTag = HYBRID_PATTERNS.find((p) => p.label.startsWith('hybrid-xss: <script>'));
|
||||||
assert.equal(scriptTag.pattern.source, '<script\\b[^>]*>');
|
assert.equal(scriptTag.pattern.source, '<script\\b[^><]*>');
|
||||||
assert.equal(scriptTag.pattern.flags, 'i');
|
assert.equal(scriptTag.pattern.flags, 'i');
|
||||||
|
|
||||||
|
const iframeSrc = HYBRID_PATTERNS.find((p) => p.label.startsWith('hybrid-xss: iframe'));
|
||||||
|
assert.equal(
|
||||||
|
iframeSrc.pattern.source,
|
||||||
|
'<iframe\\b[^><]*src\\s*=\\s*["\'][^"\']*(?:javascript:|data:text\\/html)',
|
||||||
|
);
|
||||||
|
assert.equal(iframeSrc.pattern.flags, 'i');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1373,6 +1373,79 @@ describe('injection patterns — whole-table ReDoS gate (v8.x-A)', () => {
|
||||||
`full-table sweep took ${ms.toFixed(0)}ms, budget ${TOTAL_BUDGET_MS}ms`);
|
`full-table sweep took ${ms.toFixed(0)}ms, budget ${TOTAL_BUDGET_MS}ms`);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Class 3 — the corpus above is hand-written, and a hand-written corpus is
|
||||||
|
// exactly what does not survive the next pattern. Both classes reach a
|
||||||
|
// pattern only if some unit happens to contain its leading literal, and
|
||||||
|
// `covers every exported pattern` guards the pattern LIST, not the input
|
||||||
|
// corpus — so a pattern whose prefix no unit carries is timed against input
|
||||||
|
// it rejects on the first character, and reports green having measured
|
||||||
|
// nothing. Measured 2026-08-13 on the pre-swap tables: 37 of the 45
|
||||||
|
// prefix-bearing patterns were never reached, including both quadratic
|
||||||
|
// hybrid-xss rows this gate was believed to cover.
|
||||||
|
//
|
||||||
|
// The fix is to stop hand-writing the corpus. Each pattern gets an attack
|
||||||
|
// unit built from its OWN literal prefix, so coverage is a function of the
|
||||||
|
// table rather than a list someone has to remember to extend.
|
||||||
|
//
|
||||||
|
// 64KB rather than the 512KB read cap, for the class-1 reason: a quadratic
|
||||||
|
// pattern met at 512KB stalls the run for minutes instead of failing it.
|
||||||
|
// 64KB already separates the two classes by ~700x — measured on the pre-swap
|
||||||
|
// tables, worst clean pattern 1.66ms against iframe-src 1641ms and
|
||||||
|
// script-tag 1184ms, both growing 4x per doubling.
|
||||||
|
const SELF_PREFIX_CAP = 64 * 1024;
|
||||||
|
const SELF_PREFIX_BUDGET_MS = 150;
|
||||||
|
|
||||||
|
// Leading run of plain literals, stopping at the first metacharacter or at a
|
||||||
|
// literal that a following quantifier makes optional.
|
||||||
|
const REGEX_META = new Set([...'\\^$.|?*+()[]{}']);
|
||||||
|
const literalPrefix = (source) => {
|
||||||
|
let out = '';
|
||||||
|
for (let i = 0; i < source.length; i += 1) {
|
||||||
|
if (REGEX_META.has(source[i])) break;
|
||||||
|
const next = source[i + 1];
|
||||||
|
if (next === '?' || next === '*' || next === '{') break;
|
||||||
|
out += source[i];
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
|
||||||
|
const selfPrefixProbes = ALL_PATTERNS
|
||||||
|
.map((entry) => ({ ...entry, prefix: literalPrefix(entry.pattern.source) }))
|
||||||
|
// A prefix under 2 chars is not a cheap-fail anchor: those patterns start
|
||||||
|
// on a class or group and both classes above already engage them.
|
||||||
|
.filter(({ prefix }) => prefix.length >= 2);
|
||||||
|
|
||||||
|
it('probes every pattern that has a literal prefix to fail cheaply on', () => {
|
||||||
|
// Guards this class the way `covers every exported pattern` guards the
|
||||||
|
// list — but on the axis that was actually blind. If the extractor stops
|
||||||
|
// recognising prefixes, this count collapses and says so.
|
||||||
|
assert.ok(selfPrefixProbes.length >= 40,
|
||||||
|
`expected the prefix-bearing patterns, got ${selfPrefixProbes.length} of ${ALL_PATTERNS.length}`);
|
||||||
|
for (const { pattern, label, prefix } of selfPrefixProbes) {
|
||||||
|
const unit = grow(prefix, SELF_PREFIX_CAP);
|
||||||
|
assert.ok(unit.includes(prefix),
|
||||||
|
`probe input for ${label} does not contain its own prefix ${JSON.stringify(prefix)}`);
|
||||||
|
assert.equal(unit.length, SELF_PREFIX_CAP,
|
||||||
|
`probe input for ${label} is ${unit.length} chars, expected ${SELF_PREFIX_CAP}`);
|
||||||
|
assert.ok(pattern.source.startsWith(prefix),
|
||||||
|
`extracted prefix ${JSON.stringify(prefix)} is not a prefix of ${label}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('no pattern backtracks polynomially on input built from its own prefix', () => {
|
||||||
|
assert.deepEqual(exponentialOffenders, [],
|
||||||
|
'refusing to run large input against an exponential pattern — fix the class-1 failure first');
|
||||||
|
const offenders = [];
|
||||||
|
for (const { pattern, label, prefix } of selfPrefixProbes) {
|
||||||
|
const ms = timeMs(pattern, grow(prefix, SELF_PREFIX_CAP));
|
||||||
|
if (ms > SELF_PREFIX_BUDGET_MS) {
|
||||||
|
offenders.push(`${label} :: prefix ${JSON.stringify(prefix)} :: ${ms.toFixed(0)}ms`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert.deepEqual(offenders, [],
|
||||||
|
`patterns exceeded ${SELF_PREFIX_BUDGET_MS}ms on ${SELF_PREFIX_CAP / 1024}KB of their own prefix:\n ${offenders.join('\n ')}`);
|
||||||
|
});
|
||||||
|
|
||||||
it('scanForInjection itself terminates on the pathological corpus', () => {
|
it('scanForInjection itself terminates on the pathological corpus', () => {
|
||||||
assert.deepEqual(exponentialOffenders, [],
|
assert.deepEqual(exponentialOffenders, [],
|
||||||
'refusing to run 512KB input against an exponential pattern — fix the class-1 failure first');
|
'refusing to run 512KB input against an exponential pattern — fix the class-1 failure first');
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue