1
0
Fork 0

fix(lexicon): two quadratic patterns, reachable through the output gate too

The input-path duty `8deca93` scoped. All 83 lexicon patterns measured arm by
arm; two are quadratic, same shape 0.3.2 fixed -- a run in front of a required
literal that may cross the pattern's own opening anchor. Exponent 1.98 over five
points, so quadratic, not exponential.

  markdown:link-anchor-injection  `[`          1.91s @8k   ~8.3h at the cap
  markdown:link-anchor-injection  `[system](`  0.006s @8k  ~89s at the cap
  markdown:link-ref-comment       `[//]: # (`  0.22s @8k   ~1.0h at the cap

Not input-path-only: `scan_lexicon` runs on the output path, so `scan_output("["
* 100_000)` took 334.7s. 0.3.2's "last quadratic site on the output path" was
false when written -- its sweep drove `[` only through `scan_active_content`.

Fix is anchor exclusion, not bounding (bounding attacker-controlled content is a
one-line bypass). The excluded char is `(`, not the obvious `[`: excluding `[`
drops `[//]: # (see [x] then ignore this)`, which no other pattern catches. The
anchors contain `(` too, so it telescopes at zero measured recall cost.

N is per row deliberately. The URL arm ran 0.9s UNFIXED at N=100_000 -- under the
2.0s bound, so that row could not have failed. Measured at N=300_000 instead,
where crafted (8.10s) and legitimate (0.926s) separate 8.8x.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 21:50:51 +02:00
commit b8028ba870
4 changed files with 97 additions and 5 deletions

View file

@ -588,7 +588,7 @@
},
{
"id": "markdown:link-ref-comment",
"regex": "\\[//\\]:\\s*#\\s*\\(.*(?:ignore|override|system|instruction|execute)",
"regex": "\\[//\\]:\\s*#\\s*\\([^(\\n]*(?:ignore|override|system|instruction|execute)",
"flags": "i",
"severity": "medium",
"owasp": "LLM01",
@ -604,7 +604,7 @@
},
{
"id": "markdown:link-anchor-injection",
"regex": "\\[[^\\]]*(?:system|ignore|override|exfiltrate|execute)[^\\]]*\\]\\([^)]+\\)",
"regex": "\\[[^\\]\\[]*(?:system|ignore|override|exfiltrate|execute)[^\\]\\[]*\\]\\([^)(]+\\)",
"flags": "i",
"severity": "medium",
"owasp": "LLM01",

View file

@ -23,9 +23,25 @@ caller's — this module only reports.
**Self-safety (OWASP LLM10).** A scanner that hangs on crafted input *is* the
DoS. Two guards land here (the shared guard ``entropy`` deferred to this
module): an input-size cap (:data:`MAX_SCAN_CHARS`; oversize input is scanned up
to the cap and flagged) and ReDoS-safe patterns the two sub-agent patterns
whose seed form nested ``.*?`` are ported with *bounded* token-gap quantifiers
(``(?:\\S+\\s+){0,N}?``), since Python's ``re`` has no timeout.
to the cap and flagged) and ReDoS-safe patterns, since Python's ``re`` has no
timeout. The pattern table needs *two* remedies, not one:
* **Bounded token gaps** the two sub-agent patterns whose seed form nested
``.*?`` are ported with ``(?:\\S+\\s+){0,N}?``.
* **Anchor exclusion** a run in front of a *required* literal is quadratic
whenever it may cross the pattern's own opening anchor, with no nesting
involved. Measured across all 83 patterns arm by arm, two markdown patterns
had this defect; both now exclude the anchor character from the run. The
exclusion is ``(`` rather than ``[`` in both cases: it telescopes just as
well (the anchors contain ``(`` too) and costs no measured recall, whereas
excluding ``[`` drops a link-ref comment carrying a nested bracket that no
other pattern catches. Bounding the runs instead is the wrong fix here for
the reason ``active_content`` documents the content is attacker-controlled,
so padding past a bound would be a one-line bypass.
The cap does not mitigate this on its own: it bounds the *input*, and quadratic
work on a bounded input is still hours. See
``tests/test_lexicon.py::test_crafted_redos_payload_stays_bounded_in_the_lexicon``.
The pattern table ships as JSON (``injection_lexicon.json``) the single source
of truth, decoupled from this engine for a future TS port. Non-Latin data in