fix(output): 19 quadratic regex runs on the output path, worst ~5.7h at the cap
The output gate claimed LLM10 self-safety on the grounds that its patterns have no nested quantifiers. True, and irrelevant: nesting is not what makes these blow up. A run in front of a REQUIRED literal, reachable from a short anchor, is enough -- crafted input repeats the anchor and never supplies the literal, so every start position rescans the tail. Quadratic, not exponential, and the max_scan_chars cap does not help: it bounds the input, and quadratic work on a bounded input is still hours. Measured, not argued. `<a:` x 100_000 took 23.4s in AUTOLINK_RE alone; the composed gate on that payload took 458.7s, extrapolating to ~5.7 hours at the 1_000_000-char input the gate itself accepts. Size-matched ordinary prose runs 0.31s, so the separation is 18x-660x -- unlike the blob in the neighbouring test, which is the *faster* side of prose and never exercised backtracking. Two fixes, chosen per pattern rather than uniformly: - active_content + lexicon JSON (15 runs): exclude the character that opens the pattern's own anchor (`[` for markdown, `<` for tags), so a run cannot reach past the next start position and the per-start costs telescope. Verified to cost no recall: long URLs, long alt text, and `<` inside a quoted attribute all still match. Bounding instead would have been linear too but wrong here -- the content is attacker-controlled, so padding past a bound would be a one-line bypass of the EchoLeak class this table exists to catch. - connstr egress (4 runs): bound the password at MAX_CONNSTR_VALUE. The exclusion fix is unavailable -- the anchor character is `/` and passwords containing `/` are the common case (measured: they match today). The residual miss is a credential over 256 chars; a token that long is still caught by egress:jwt-token. hybrid-xss:script-tag had neither option: its run is the script BODY, which may legitimately contain `<`. It now matches the opening tag and drops the `</script>` requirement. That also closes a fail-open -- `<script>alert(1)` unclosed was silently missed -- at the cost of flagging prose that merely mentions `<script>`, now documented. Found by the composed-gate test staying red after every individual scanner was already linear: the lexicon's six html-obfuscation patterns were the remaining 813x. A per-scanner test alone would have shipped that. 662 passed (was 642), and faster than before the fix.
This commit is contained in:
parent
8deca93ee1
commit
cff043787d
10 changed files with 220 additions and 30 deletions
|
|
@ -244,6 +244,22 @@ items; this is the full list, each with the mechanism.
|
|||
only, so hex (and other encodings, or nested wraps) is a deliberate boundary —
|
||||
decode the transport layer first if you need it scanned.
|
||||
|
||||
- **Prose that merely mentions `<script>` fires `hybrid-xss:script-tag`.** The
|
||||
pattern matches the opening tag and no longer requires `</script>`, so a
|
||||
document *about* XSS is flagged alongside a document that *carries* it. This
|
||||
is a deliberate trade made twice over: requiring the closing tag was a
|
||||
fail-open (an unclosed `<script>alert(1)` was silently missed) and it was the
|
||||
last quadratic-backtracking site on the output path. Report-only, so the cost
|
||||
is a review, not a block.
|
||||
|
||||
- **A connection-string password longer than 256 chars is not matched.** The
|
||||
password run in the `*-connstr` egress patterns is bounded by
|
||||
`MAX_CONNSTR_VALUE`; unbounded, it sits in front of a mandatory `@` and makes
|
||||
crafted input quadratic. Excluding the anchor character instead — the fix the
|
||||
active-content table uses — is unavailable here because that character is `/`,
|
||||
and a password containing `/` is the common case. The realistic long value (a
|
||||
token used as a DB password) is still caught by `egress:jwt-token`.
|
||||
|
||||
## The six documented gaps (tracked by the coverage matrix)
|
||||
|
||||
These are asserted to *still hold* by `tests/test_coverage_matrix.py` — a closed gap
|
||||
|
|
|
|||
18
docs/PLAN.md
18
docs/PLAN.md
|
|
@ -132,11 +132,19 @@ Maximal reuse: most detection logic is a JS→Python **port**, not new code.
|
|||
- **Contract asserters** — a tool-carrying request and a credential-leaking stage env both
|
||||
raise; the happy path passes.
|
||||
- **Self-safety** — pathological/ReDoS-prone and oversize input return within a bound,
|
||||
never hang. Scope, measured 2026-07-31: the ReDoS half is carried by the *lexicon*
|
||||
path alone (`test_redos_pathological_subagent_input_returns_fast`, crafted against a
|
||||
known-bad nested `.*?`). The `output` path's bound is a no-hang guard only — its blob
|
||||
is *slower* than size-matched ordinary prose (0.93x/0.96x), so it does not exercise
|
||||
catastrophic backtracking. A crafted payload for the output regexes is not written.
|
||||
never hang. Scope, measured 2026-07-31: the *lexicon* path is covered by
|
||||
`test_redos_pathological_subagent_input_returns_fast` (crafted against a known-bad
|
||||
nested `.*?`). The `output` path is covered by
|
||||
`test_crafted_redos_payload_stays_bounded` — 15 crafted payloads plus one through the
|
||||
composed gate. Writing them found the defect they were meant to rule out: 19 quadratic
|
||||
runs across 17 patterns in `output` (4), `active_content` (5 patterns / 7 runs) and the
|
||||
lexicon JSON (8), worst case ~5.7 hours at the 1_000_000-char cap the gate accepts. Note the shape, since it is
|
||||
*not* the textbook one: no nested quantifier is involved. A run in front of a required
|
||||
literal, reachable from a short anchor, is enough — crafted input repeats the anchor,
|
||||
never supplies the literal, and every start position rescans the tail. The earlier
|
||||
"output blob is slower than ordinary prose (0.93x/0.96x)" measurement stands and was
|
||||
never wrong; it simply measured throughput on a blob, which is a different question
|
||||
from what a crafted payload asks.
|
||||
- **Neutralize** — active-content output is defanged; clean output is byte-identical.
|
||||
- **End-to-end showcase (the FINAL deliverable, built last).** One realistic
|
||||
piece of ingested content that carries *many* vulnerabilities at once — visible
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue