fix(output): 19 quadratic regex runs on the output path, worst ~5.7h at the cap
The output gate claimed LLM10 self-safety on the grounds that its patterns have no nested quantifiers. True, and irrelevant: nesting is not what makes these blow up. A run in front of a REQUIRED literal, reachable from a short anchor, is enough -- crafted input repeats the anchor and never supplies the literal, so every start position rescans the tail. Quadratic, not exponential, and the max_scan_chars cap does not help: it bounds the input, and quadratic work on a bounded input is still hours. Measured, not argued. `<a:` x 100_000 took 23.4s in AUTOLINK_RE alone; the composed gate on that payload took 458.7s, extrapolating to ~5.7 hours at the 1_000_000-char input the gate itself accepts. Size-matched ordinary prose runs 0.31s, so the separation is 18x-660x -- unlike the blob in the neighbouring test, which is the *faster* side of prose and never exercised backtracking. Two fixes, chosen per pattern rather than uniformly: - active_content + lexicon JSON (15 runs): exclude the character that opens the pattern's own anchor (`[` for markdown, `<` for tags), so a run cannot reach past the next start position and the per-start costs telescope. Verified to cost no recall: long URLs, long alt text, and `<` inside a quoted attribute all still match. Bounding instead would have been linear too but wrong here -- the content is attacker-controlled, so padding past a bound would be a one-line bypass of the EchoLeak class this table exists to catch. - connstr egress (4 runs): bound the password at MAX_CONNSTR_VALUE. The exclusion fix is unavailable -- the anchor character is `/` and passwords containing `/` are the common case (measured: they match today). The residual miss is a credential over 256 chars; a token that long is still caught by egress:jwt-token. hybrid-xss:script-tag had neither option: its run is the script BODY, which may legitimately contain `<`. It now matches the opening tag and drops the `</script>` requirement. That also closes a fail-open -- `<script>alert(1)` unclosed was silently missed -- at the cost of flagging prose that merely mentions `<script>`, now documented. Found by the composed-gate test staying red after every individual scanner was already linear: the lexicon's six html-obfuscation patterns were the remaining 813x. A per-scanner test alone would have shipped that. 662 passed (was 642), and faster than before the fix.
This commit is contained in:
parent
8deca93ee1
commit
cff043787d
10 changed files with 220 additions and 30 deletions
|
|
@ -104,31 +104,52 @@ def redact(s: str, show_start: int = 16, show_end: int = 6) -> str:
|
|||
|
||||
|
||||
# --- active-content constructs (the shared pattern table) ---------------------
|
||||
#
|
||||
# ReDoS note (OWASP LLM10) — every run below excludes the character that OPENS
|
||||
# this pattern's own anchor: `[` for the markdown forms, `<` for the autolink and
|
||||
# the raw tag. That exclusion is what keeps the table linear, and it is not
|
||||
# cosmetic. Each of these is a run followed by a REQUIRED literal (`]`, `)`,
|
||||
# `>`); if the run may cross the next anchor, then crafted input that repeats the
|
||||
# anchor and never supplies the literal makes every start position rescan the
|
||||
# whole tail — quadratic time, no nested quantifier needed. Measured before the
|
||||
# exclusions: `<a:` x 100_000 took 23.4s in AUTOLINK_RE alone and ~5.7 HOURS
|
||||
# extrapolated to the 1_000_000-char cap the gate accepts. With the exclusion a
|
||||
# run cannot reach past the next anchor, so the per-start costs telescope.
|
||||
# Bounding the runs instead ({0,256}) would also be linear but is the WRONG fix
|
||||
# here: the content is attacker-controlled, so padding past the bound would be a
|
||||
# one-line detection bypass of the very EchoLeak class this table exists to
|
||||
# catch. See tests/test_output.py::test_crafted_redos_payload_stays_bounded.
|
||||
#
|
||||
# Markdown image / inline link: `[text](url "title")`. `url` stops at the first
|
||||
# `)` or whitespace (balanced-paren URLs matched conservatively — see the
|
||||
# neutralize scope note).
|
||||
# neutralize scope note). `[` is excluded per the ReDoS note above; a URL that
|
||||
# needs a literal `[` (an IPv6 host literal) must percent-encode it anyway.
|
||||
MD_IMAGE_RE = re.compile(
|
||||
r"!\[(?P<alt>[^\]]*)\]\(\s*(?P<url>[^)\s]+)(?P<title>(?:\s+\"[^\"]*\")?)\s*\)"
|
||||
r"!\[(?P<alt>[^\]\[]*)\]\(\s*(?P<url>[^)\s\[]+)(?P<title>(?:\s+\"[^\"]*\")?)\s*\)"
|
||||
)
|
||||
MD_LINK_RE = re.compile(
|
||||
r"(?<!!)\[(?P<text>[^\]]*)\]\(\s*(?P<url>[^)\s]+)(?P<title>(?:\s+\"[^\"]*\")?)\s*\)"
|
||||
r"(?<!!)\[(?P<text>[^\]\[]*)\]\(\s*(?P<url>[^)\s\[]+)(?P<title>(?:\s+\"[^\"]*\")?)\s*\)"
|
||||
)
|
||||
# Reference-style link definition: `[label]: destination`. Only fires when the
|
||||
# destination is absolute (has a scheme or is protocol-relative) — a footnote
|
||||
# `[1]: some plain text` is not a link target and is left alone.
|
||||
MD_REFDEF_RE = re.compile(
|
||||
r"(?m)^(?P<pre>[ ]{0,3}\[[^\]]+\]:\s*)(?P<url>[A-Za-z][\w+.\-]*:\S+|//\S+)"
|
||||
r"(?m)^(?P<pre>[ ]{0,3}\[[^\]\[]+\]:\s*)(?P<url>[A-Za-z][\w+.\-]*:\S+|//\S+)"
|
||||
)
|
||||
# Angle-bracket autolink: `<scheme:...>`.
|
||||
AUTOLINK_RE = re.compile(r"<(?P<url>[A-Za-z][A-Za-z0-9+.\-]*:[^>\s]+)>")
|
||||
# Angle-bracket autolink: `<scheme:...>`. A URL inside `<...>` cannot contain a
|
||||
# raw `<`, so excluding it costs no recall (verified) and bounds the run.
|
||||
AUTOLINK_RE = re.compile(r"<(?P<url>[A-Za-z][A-Za-z0-9+.\-]*:[^>\s<]+)>")
|
||||
# Standalone `data:` URI in prose (not preceded by a letter/digit -> "metadata:"
|
||||
# is not a match), consuming to the next whitespace / quote / bracket.
|
||||
DATA_URI_RE = re.compile(r"(?<![A-Za-z0-9])data:[^\s'\"<>)]+", re.IGNORECASE)
|
||||
|
||||
# Raw HTML tag. Attribute values may hold `>` inside quotes, so quoted runs are
|
||||
# consumed atomically. A tag is *active* if it is an inherently-executing element,
|
||||
# carries an event handler, or carries a URL-bearing attribute.
|
||||
HTML_TAG_RE = re.compile(r"<(?P<slash>/?)(?P<name>[A-Za-z][A-Za-z0-9:-]*)(?P<attrs>(?:[^>\"']|\"[^\"]*\"|'[^']*')*)>")
|
||||
# carries an event handler, or carries a URL-bearing attribute. The unquoted-char
|
||||
# branch excludes `<` per the ReDoS note above — a raw `<` cannot appear in an
|
||||
# unquoted attribute region anyway, and a `<` inside a QUOTED value is still
|
||||
# consumed by the quoted branches, so this costs no recall (verified).
|
||||
HTML_TAG_RE = re.compile(r"<(?P<slash>/?)(?P<name>[A-Za-z][A-Za-z0-9:-]*)(?P<attrs>(?:[^>\"'<]|\"[^\"]*\"|'[^']*')*)>")
|
||||
_EVENT_ATTR_RE = re.compile(r"\bon[a-z]+\s*=", re.IGNORECASE)
|
||||
_URL_ATTR_RE = re.compile(
|
||||
r"\b(?:src|href|xlink:href|srcset|data|poster|formaction|action|background|cite|codebase|longdesc)\s*=",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue