1
0
Fork 0

fix(output): 19 quadratic regex runs on the output path, worst ~5.7h at the cap

The output gate claimed LLM10 self-safety on the grounds that its patterns have
no nested quantifiers. True, and irrelevant: nesting is not what makes these
blow up. A run in front of a REQUIRED literal, reachable from a short anchor, is
enough -- crafted input repeats the anchor and never supplies the literal, so
every start position rescans the tail. Quadratic, not exponential, and the
max_scan_chars cap does not help: it bounds the input, and quadratic work on a
bounded input is still hours.

Measured, not argued. `<a:` x 100_000 took 23.4s in AUTOLINK_RE alone; the
composed gate on that payload took 458.7s, extrapolating to ~5.7 hours at the
1_000_000-char input the gate itself accepts. Size-matched ordinary prose runs
0.31s, so the separation is 18x-660x -- unlike the blob in the neighbouring
test, which is the *faster* side of prose and never exercised backtracking.

Two fixes, chosen per pattern rather than uniformly:

- active_content + lexicon JSON (15 runs): exclude the character that opens the
  pattern's own anchor (`[` for markdown, `<` for tags), so a run cannot reach
  past the next start position and the per-start costs telescope. Verified to
  cost no recall: long URLs, long alt text, and `<` inside a quoted attribute
  all still match. Bounding instead would have been linear too but wrong here --
  the content is attacker-controlled, so padding past a bound would be a
  one-line bypass of the EchoLeak class this table exists to catch.
- connstr egress (4 runs): bound the password at MAX_CONNSTR_VALUE. The
  exclusion fix is unavailable -- the anchor character is `/` and passwords
  containing `/` are the common case (measured: they match today). The residual
  miss is a credential over 256 chars; a token that long is still caught by
  egress:jwt-token.

hybrid-xss:script-tag had neither option: its run is the script BODY, which may
legitimately contain `<`. It now matches the opening tag and drops the
`</script>` requirement. That also closes a fail-open -- `<script>alert(1)`
unclosed was silently missed -- at the cost of flagging prose that merely
mentions `<script>`, now documented.

Found by the composed-gate test staying red after every individual scanner was
already linear: the lexicon's six html-obfuscation patterns were the remaining
813x. A per-scanner test alone would have shipped that.

662 passed (was 642), and faster than before the fix.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 18:31:58 +02:00
commit cff043787d
10 changed files with 220 additions and 30 deletions

View file

@ -36,6 +36,10 @@ def test_lexicon_selfsafety_frozen():
assert cal.ROT13_MIN_LEN == 40
def test_output_selfsafety_frozen():
assert cal.MAX_CONNSTR_VALUE == 256
def test_cognitive_load_lengths_frozen():
assert cal.COGNITIVE_LOAD_MIN_LEN == 2500
assert cal.COGNITIVE_LOAD_TAIL_START == 2000
@ -102,6 +106,18 @@ def test_lexicon_module_sources_from_calibration():
assert lexicon._ROT13_MIN_LEN is cal.ROT13_MIN_LEN
def test_output_module_sources_from_calibration():
# The bound is baked into the compiled patterns, so `is` on a module
# attribute cannot prove sourcing here -- assert the compiled regex carries
# the calibrated number instead.
from llm_ingestion_guard import output
assert output.MAX_CONNSTR_VALUE is cal.MAX_CONNSTR_VALUE
connstr = [p for p in output._SECRET_PATTERNS if p.id.endswith("-connstr")]
assert len(connstr) == 4
for pattern in connstr:
assert f"{{1,{cal.MAX_CONNSTR_VALUE}}}" in pattern.regex.pattern
def test_disposition_module_sources_from_calibration():
from llm_ingestion_guard import disposition
from llm_ingestion_guard.disposition import Disposition