fix(lexicon): two quadratic patterns, reachable through the output gate too
The input-path duty `8deca93` scoped. All 83 lexicon patterns measured arm by
arm; two are quadratic, same shape 0.3.2 fixed -- a run in front of a required
literal that may cross the pattern's own opening anchor. Exponent 1.98 over five
points, so quadratic, not exponential.
markdown:link-anchor-injection `[` 1.91s @8k ~8.3h at the cap
markdown:link-anchor-injection `[system](` 0.006s @8k ~89s at the cap
markdown:link-ref-comment `[//]: # (` 0.22s @8k ~1.0h at the cap
Not input-path-only: `scan_lexicon` runs on the output path, so `scan_output("["
* 100_000)` took 334.7s. 0.3.2's "last quadratic site on the output path" was
false when written -- its sweep drove `[` only through `scan_active_content`.
Fix is anchor exclusion, not bounding (bounding attacker-controlled content is a
one-line bypass). The excluded char is `(`, not the obvious `[`: excluding `[`
drops `[//]: # (see [x] then ignore this)`, which no other pattern catches. The
anchors contain `(` too, so it telescopes at zero measured recall cost.
N is per row deliberately. The URL arm ran 0.9s UNFIXED at N=100_000 -- under the
2.0s bound, so that row could not have failed. Measured at N=300_000 instead,
where crafted (8.10s) and legitimate (0.926s) separate 8.8x.
This commit is contained in:
parent
75ae48277b
commit
b8028ba870
4 changed files with 97 additions and 5 deletions
|
|
@ -588,7 +588,7 @@
|
|||
},
|
||||
{
|
||||
"id": "markdown:link-ref-comment",
|
||||
"regex": "\\[//\\]:\\s*#\\s*\\(.*(?:ignore|override|system|instruction|execute)",
|
||||
"regex": "\\[//\\]:\\s*#\\s*\\([^(\\n]*(?:ignore|override|system|instruction|execute)",
|
||||
"flags": "i",
|
||||
"severity": "medium",
|
||||
"owasp": "LLM01",
|
||||
|
|
@ -604,7 +604,7 @@
|
|||
},
|
||||
{
|
||||
"id": "markdown:link-anchor-injection",
|
||||
"regex": "\\[[^\\]]*(?:system|ignore|override|exfiltrate|execute)[^\\]]*\\]\\([^)]+\\)",
|
||||
"regex": "\\[[^\\]\\[]*(?:system|ignore|override|exfiltrate|execute)[^\\]\\[]*\\]\\([^)(]+\\)",
|
||||
"flags": "i",
|
||||
"severity": "medium",
|
||||
"owasp": "LLM01",
|
||||
|
|
|
|||
|
|
@ -23,9 +23,25 @@ caller's — this module only reports.
|
|||
**Self-safety (OWASP LLM10).** A scanner that hangs on crafted input *is* the
|
||||
DoS. Two guards land here (the shared guard ``entropy`` deferred to this
|
||||
module): an input-size cap (:data:`MAX_SCAN_CHARS`; oversize input is scanned up
|
||||
to the cap and flagged) and ReDoS-safe patterns — the two sub-agent patterns
|
||||
whose seed form nested ``.*?`` are ported with *bounded* token-gap quantifiers
|
||||
(``(?:\\S+\\s+){0,N}?``), since Python's ``re`` has no timeout.
|
||||
to the cap and flagged) and ReDoS-safe patterns, since Python's ``re`` has no
|
||||
timeout. The pattern table needs *two* remedies, not one:
|
||||
|
||||
* **Bounded token gaps** — the two sub-agent patterns whose seed form nested
|
||||
``.*?`` are ported with ``(?:\\S+\\s+){0,N}?``.
|
||||
* **Anchor exclusion** — a run in front of a *required* literal is quadratic
|
||||
whenever it may cross the pattern's own opening anchor, with no nesting
|
||||
involved. Measured across all 83 patterns arm by arm, two markdown patterns
|
||||
had this defect; both now exclude the anchor character from the run. The
|
||||
exclusion is ``(`` rather than ``[`` in both cases: it telescopes just as
|
||||
well (the anchors contain ``(`` too) and costs no measured recall, whereas
|
||||
excluding ``[`` drops a link-ref comment carrying a nested bracket that no
|
||||
other pattern catches. Bounding the runs instead is the wrong fix here for
|
||||
the reason ``active_content`` documents — the content is attacker-controlled,
|
||||
so padding past a bound would be a one-line bypass.
|
||||
|
||||
The cap does not mitigate this on its own: it bounds the *input*, and quadratic
|
||||
work on a bounded input is still hours. See
|
||||
``tests/test_lexicon.py::test_crafted_redos_payload_stays_bounded_in_the_lexicon``.
|
||||
|
||||
The pattern table ships as JSON (``injection_lexicon.json``) — the single source
|
||||
of truth, decoupled from this engine for a future TS port. Non-Latin data in
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue