fix(sanitize,okf,active_content): three quadratic patterns, two on the input path

The generalised sweep found what 0.3.2's hand-written rows missed. All three are
the documented class -- a run in front of a required literal that never arrives,
so every start position rescans the tail -- and all three are worse than the
0.3.3 findings, because `sanitize`, `neutralize`, `scan_active_content` and the
okf link graph apply NO input cap. `scan_lexicon`/`scan_output` are the only
entry points that do, so there is no ceiling to extrapolate to.

  sanitize._HTML_COMMENT_RE   `<!--`*100_000        20.1s, exponent 1.96-2.14
  active_content.URL_IN_TEXT_RE  `<a `+`A`*100_000  12.99s / 14.9s, exponent ~2.0
  okf._MD_LINK_RE             `[`*100_000            7.1s, exponent 1.99-2.05

Each fix is the one the pattern's own shape allows, not a copied choice:

  - The comment stripper drops the regex for `str.find`. Excluding `<` would lose
    every comment containing markup; bounding the run would be a carrier bypass
    of the exact construct the stripper exists to remove.
  - `URL_IN_TEXT_RE` bounds its scheme run to an RFC 3986 scheme (`{0,63}`).
    Bounding is safe *here* only because it is a defanger inside a tag already
    flagged `active:raw-html`. A lookbehind was measured too and rejected: it
    drops `-http://evil.com`, a one-character evasion. Bounded: 0.185s at 1M.
  - `_MD_LINK_RE` excludes `[`, matching `active_content.MD_LINK_RE` exactly,
    including the nested-label trade already documented there.

`sanitize` claimed "no catastrophic backtracking" in a comment; that claim was
wrong in the same way `output`'s was before 0.3.2, and is corrected in place.

676 tests (+10), coverage 128/128 + 6/6 gaps, sweep clean across 150 patterns.
The okf destination run gets no row: `[^)\s]+` cannot fail, so a row for it
could never go red.

docs/redos-sweep.py

@@ -249,8 +249,11 @@ def collect_output() -> list[Target]:
 TABLES = {
     "lexicon": collect_lexicon,
     # The normalizers run `.sub()` over EVERY input before any pattern matches;
-    # 0.3.3 swept the 83 patterns and not these.
-    "normalize": lambda: collect_module("lexicon", "normalize"),
+    # 0.3.3 swept the 83 patterns and not these. `_LEXICON_CACHE` is skipped: it
+    # is empty until `load_lexicon()` runs, so counting it would make this
+    # table's size depend on whether `lexicon` was swept first.
+    "normalize": lambda: collect_module("lexicon", "normalize",
+                                        skip={"_LEXICON_CACHE"}),
     "active_content": lambda: collect_module("active_content", "active_content"),
     "entropy": lambda: collect_module("entropy", "entropy") + [
         # Inline literals in is_base64_like / is_hex_blob (entropy.py:111,118),

src/llm_ingestion_guard/active_content.py

@@ -80,7 +80,26 @@ _SCHEME_SUBS = (
 # Dot-defang that is idempotent: never touches a `.` already inside `[.]`.
 _DOT_RE = re.compile(r"(?<!\[)\.(?!\])")
 # A bare http(s)/ftp URL embedded in other text (used inside escaped HTML).
-URL_IN_TEXT_RE = re.compile(r"[A-Za-z][A-Za-z0-9+.\-]*://[^\s'\"<>]+")
+#
+# ReDoS note (OWASP LLM10). The scheme run sits in front of a REQUIRED `://`, so
+# a long run of scheme characters that never reaches it costs a full rescan at
+# every start position: `<a ` + `A`*100_000 + `>` measured 12.99s through
+# `scan_active_content` and 14.9s through `neutralize`, exponent ~2.0 over four
+# doublings, on entry points that apply no input cap. The 0.3.2 sweep missed it
+# because its payloads repeat a unit, and this arm needs the tag to CLOSE before
+# the body is handed on.
+#
+# The exclusion trick used by the constructs below does not apply — the attack
+# repeats a plain scheme character, not this pattern's anchor — so the run is
+# bounded to an RFC 3986 scheme instead (`ALPHA *( ALPHA / DIGIT / "+" / "-" /
+# "." )`; the longest registered scheme is far under 64). Unlike the detector
+# tables, bounding costs nothing here: this is a defanger applied INSIDE a tag
+# already flagged `active:raw-html`, padding merely shifts where the match
+# starts, and a 64+ character "scheme" is not resolvable by any renderer. A
+# lookbehind that killed interior start positions was measured too and rejected:
+# it drops `-http://evil.com` and `.http://x.com`, a one-character evasion of
+# the defanger. Bounded: 0.185s at the full 1_000_000-char cap.
+URL_IN_TEXT_RE = re.compile(r"[A-Za-z][A-Za-z0-9+.\-]{0,63}://[^\s'\"<>]+")
 
 
 def defang_url(url: str) -> str:

src/llm_ingestion_guard/okf.py

@@ -407,7 +407,14 @@ def _most_severe(dispositions):
 # consumer that owns the corpus decides where the durable graph state lives.
 # This in-import graph resolves links within a single bundle merge.
 
-_MD_LINK_RE = re.compile(r"\[[^\]]*\]\(\s*([^)\s]+)")
+# ReDoS note (OWASP LLM10): the label run excludes `[`, the character that opens
+# this pattern's own anchor. Without it, a bundle body repeating `[` and never
+# closing it makes every start position rescan the tail — 7.1s at 100_000 chars,
+# exponent ~2.0, over attacker-supplied bodies this adapter reads with no input
+# cap. Same defect and same fix as `active_content.MD_LINK_RE`, including the
+# trade it names: a label containing a nested `[...]` is given up on, which costs
+# no exfil coverage because the inner link is matched on its own.
+_MD_LINK_RE = re.compile(r"\[[^\]\[]*\]\(\s*([^)\s]+)")
 # Active-content schemes are refused in a link, mirroring the resource gate (T3).
 _DANGEROUS_LINK_SCHEMES = frozenset({"javascript", "data", "vbscript", "file", "blob"})
 

src/llm_ingestion_guard/sanitize.py

@@ -22,8 +22,24 @@ _ZERO_WIDTH = frozenset({0x200B, 0x200C, 0x200D, 0xFEFF, 0x00AD})
 _BIDI = frozenset({0x202A, 0x202B, 0x202C, 0x202D, 0x202E, 0x2066, 0x2067, 0x2068, 0x2069})
 _TAG_LO, _TAG_HI = 0xE0000, 0xE007F  # Unicode Tags block (U+E0000–U+E007F)
 
-# Span carriers. Lazy `.*?` + explicit terminator — no catastrophic backtracking.
-_HTML_COMMENT_RE = re.compile(r"<!--.*?-->", re.DOTALL)
+# Span carriers.
+#
+# ReDoS note (OWASP LLM10). The comment stripper used to be `<!--.*?-->` with a
+# comment that absence of nesting meant no catastrophic backtracking. That claim
+# was wrong in the same way `output`'s was: a lazy run in front of a REQUIRED
+# literal costs a full tail rescan at *every* start position when the literal
+# never arrives, so `<!--` repeated to 100_000 chars measured 20.1s (exponent
+# 1.96–2.14 over four doublings) — and this module, unlike `scan_lexicon` /
+# `scan_output`, applies no input cap, so nothing bounds that above.
+#
+# Neither of the two fixes used elsewhere fits here. Excluding the opener (`<`)
+# from the run would drop every comment containing markup — `<!-- <b>x</b> -->`
+# is the ordinary case, not an edge one. Bounding the run would be a one-line
+# carrier bypass: a comment padded past the bound is exactly what this stripper
+# exists to remove. So the scan is done with `str.find`, which is linear and
+# semantically identical to the lazy regex — leftmost opener, nearest following
+# terminator, unterminated trailer left in place.
+_COMMENT_OPEN, _COMMENT_CLOSE = "<!--", "-->"
 # `data:` not preceded by a letter (so "metadata:" / "userdata:" do not match),
 # consuming up to the next whitespace / quote / angle bracket / closing paren.
 _DATA_URI_RE = re.compile(r"(?<![A-Za-z])data:[^\s'\"<>)]+", re.IGNORECASE)
@@ -43,6 +59,32 @@ def _redact(s: str, show_start: int = 12, show_end: int = 4) -> str:
     return f"{s[:show_start]}...{s[-show_end:]}"
 
 
+def _strip_html_comments(text: str) -> tuple[str, int]:
+    """Remove ``<!-- ... -->`` spans; return the cleaned text and the count.
+
+    Linear replacement for the quantifier form (see the ReDoS note above). An
+    unterminated ``<!--`` is left verbatim, matching the regex it replaces.
+    """
+    if _COMMENT_OPEN not in text:
+        return text, 0
+    out: list[str] = []
+    pos = count = 0
+    while True:
+        start = text.find(_COMMENT_OPEN, pos)
+        if start == -1:
+            break
+        end = text.find(_COMMENT_CLOSE, start + len(_COMMENT_OPEN))
+        if end == -1:  # unterminated — not a comment, keep the rest verbatim
+            break
+        out.append(text[pos:start])
+        pos = end + len(_COMMENT_CLOSE)
+        count += 1
+    if not count:
+        return text, 0
+    out.append(text[pos:])
+    return "".join(out), count
+
+
 def _decode_tags(codepoints: list[int]) -> str:
     """Decode Unicode-tag codepoints to their hidden ASCII (cp - 0xE0000)."""
     out = []
@@ -75,7 +117,7 @@ def sanitize(text: str, source: Source = Source.INPUT) -> SanitizeResult:
     cleaned = "".join(kept) if (zero_width or bidi or tag_cps) else text
 
     # Span carriers.
-    cleaned, n_comments = _HTML_COMMENT_RE.subn("", cleaned)
+    cleaned, n_comments = _strip_html_comments(cleaned)
     cleaned, n_data = _DATA_URI_RE.subn("", cleaned)
 
     if zero_width:

tests/test_active_content.py

@@ -18,6 +18,8 @@ from __future__ import annotations
 
 import pytest
 
+import time
+
 from llm_ingestion_guard import (
     scan_active_content,
     scan_output,
@@ -294,3 +296,38 @@ def test_raw_html_counts_end_tags():
             if f.label == "active:raw-html"]
     assert len(pair) == 1, "a start/end pair must not split into two findings"
     assert pair[0].count == 2, f"end tag not counted: {pair[0].count}"
+
+
+# --- self-safety (OWASP LLM10): the long-attribute arm -----------------------
+# The `_REDOS_PAYLOADS` rows in test_output.py attack tags that never CLOSE, so
+# `HTML_TAG_RE` fails and the tag body is never handed on. This arm is the
+# opposite: the tag closes, and its body is long. `_tag` then runs
+# `URL_IN_TEXT_RE` over it, whose scheme run sits in front of a required `://`
+# that never arrives — 12.99s at 100_000 chars through this scanner, exponent
+# 1.87-2.06 over four doublings, with no input cap on this entry point at all.
+# Missed by the 0.3.2 sweep because a repeating-unit payload cannot express
+# "one tag, long body"; found by docs/redos-sweep.py generalised past lexicon.
+_ATTR_REDOS_N = 100_000
+
+
+def test_crafted_long_attribute_tag_stays_bounded():
+    payload = "<a " + "A" * _ATTR_REDOS_N + ">"
+    start = time.monotonic()
+    scan_active_content(payload)
+    assert time.monotonic() - start < 2.0
+
+
+def test_url_defanging_survives_the_redos_fix():
+    # Recall parity for the evidence defanger, including the two forms a
+    # lookbehind-based fix would have dropped (`-` / `.` immediately before the
+    # scheme), which is why the scheme run is bounded instead.
+    for raw, expected in (
+        ("<a href=http://evil.com>", "hxxp"),
+        ("<a href=-http://evil.com>", "hxxp"),
+        ("<a href=.http://x.com>", "hxxp"),
+        ('<a href="https://a.b/c">', "hxxps"),
+    ):
+        report = scan_active_content(raw)
+        evidence = " ".join(f.evidence or "" for f in report.findings)
+        assert expected in evidence, raw
+        assert "http://" not in evidence and "https://" not in evidence, raw

tests/test_neutralize.py

@@ -11,6 +11,8 @@ empty report; only active-content constructs are ever rewritten. Mutation lives
 here, kept separate from the report-only output gate (design principles 3 & 4).
 The transform is pure ``text -> (defanged_text, report)`` — no I/O, no globals.
 """
+import time
+
 from llm_ingestion_guard.neutralize import neutralize
 from llm_ingestion_guard.report import Severity, Source
 
@@ -139,3 +141,23 @@ def test_prose_with_lone_brackets_and_angles_is_identical():
     result = neutralize(text)
     assert result.text == text
     assert result.report.found is False
+
+
+# --- self-safety (OWASP LLM10): the long-attribute arm -----------------------
+# Second call site of the same defect pinned in test_active_content.py: the
+# defanger runs `URL_IN_TEXT_RE` over each active tag's body. 14.9s at 100_000
+# chars, exponent 1.91-2.22. `neutralize` applies no input cap either.
+_ATTR_REDOS_N = 100_000
+
+
+def test_crafted_long_attribute_tag_stays_bounded():
+    payload = "<a " + "A" * _ATTR_REDOS_N + ">"
+    start = time.monotonic()
+    neutralize(payload)
+    assert time.monotonic() - start < 2.0
+
+
+def test_url_defanging_inside_a_tag_survives_the_redos_fix():
+    result = neutralize("<a href=-http://evil.com>x</a>")
+    assert "hxxp" in result.text
+    assert "http://evil.com" not in result.text

tests/test_okf.py

@@ -17,6 +17,8 @@ OKF spec facts used here (verified against okf/SPEC.md, 2026-07-06):
 """
 import pytest
 
+import time
+
 from llm_ingestion_guard.okf import (
     parse_frontmatter,
     scan_concept,
@@ -640,3 +642,34 @@ def test_t2_constrains_import_not_emission(fm):
     doc = f"---\nid: x\n{fm}---\n\nbody\n"
     assert import_bundle({"concepts/x.md": doc}).disposition is Disposition.FAIL_SECURE
     assert screen_output(doc, PRESET_USER_UPLOAD).disposition is Disposition.WARN
+
+
+# --- self-safety (OWASP LLM10): ReDoS in the link-graph extractor ------------
+# `[^\]]*` is a run in front of a REQUIRED `]`: a bundle body that repeats `[`
+# and never closes it makes every start position rescan the tail. Measured 7.1s
+# at 100_000 chars, exponent 1.99-2.05 over four doublings, and the link graph
+# runs over attacker-supplied bundle bodies with no input cap. Found by
+# docs/redos-sweep.py once it was generalised past the lexicon table; the same
+# defect in `active_content`'s markdown table was already fixed there the same
+# way, by excluding the character that opens the pattern's own anchor.
+_LINK_REDOS_N = 100_000
+
+
+def test_crafted_link_payload_stays_bounded():
+    start = time.monotonic()
+    extract_link_targets("[" * _LINK_REDOS_N)
+    assert time.monotonic() - start < 2.0
+
+
+# The destination run behind the label gets no row: `[^)\s]+` needs only one
+# character, so it cannot fail, and a run that cannot fail cannot pay the
+# per-start rescan. A row for it could never go red — decoration, not a pin.
+
+
+def test_link_extraction_survives_the_redos_fix():
+    # Recall parity: ordinary links, a label holding brackets it does not close,
+    # and the nested-bracket form the exclusion deliberately gives up on -- the
+    # same trade `active_content.MD_LINK_RE` already makes.
+    assert extract_link_targets("see [x](./a.md) and [y](/b.md)") == ["./a.md", "/b.md"]
+    assert extract_link_targets("[a b](./c.md)") == ["./c.md"]
+    assert extract_link_targets("text [![img](./i.png)](./t.md)") == ["./i.png"]

tests/test_output.py

@@ -436,3 +436,15 @@ def test_gate_is_bounded_on_the_payload_the_first_sweep_missed():
     start = time.monotonic()
     scan_output(payload)
     assert time.monotonic() - start < 2.0
+
+
+def test_gate_is_bounded_on_the_long_attribute_arm():
+    # The composed-gate row for the defect pinned in test_active_content.py and
+    # test_neutralize.py. `scan_output` runs `scan_active_content`, so the gate a
+    # caller actually invokes inherits it. Not expressible as a repeating unit —
+    # the tag has to CLOSE for the body to be handed on — which is exactly why
+    # the unit-table above never covered it.
+    payload = "<a " + "A" * _REDOS_N + ">"
+    start = time.monotonic()
+    scan_output(payload)
+    assert time.monotonic() - start < 2.0

tests/test_sanitize.py

@@ -4,6 +4,8 @@ Core invariants (BRIEF §9): clean input returns byte-identical with an all-zero
 report; the sanitizer only ever *removes* — its output is always a subsequence
 of the input.
 """
+import time
+
 from llm_ingestion_guard.sanitize import sanitize
 from llm_ingestion_guard.report import Severity, Source
 
@@ -75,3 +77,42 @@ def test_data_uri_does_not_match_inside_a_word():
 def test_output_source_is_respected():
     result = sanitize("x​y", source=Source.OUTPUT)
     assert all(f.source is Source.OUTPUT for f in result.report.findings)
+
+
+# --- self-safety (OWASP LLM10): ReDoS on the comment stripper ----------------
+# `sanitize` is step 1 of `prepare_input` -- the first thing every ingested
+# document hits -- and unlike `scan_lexicon`/`scan_output` it applies NO input
+# cap, so a quadratic run here has no ceiling at all. `<!--.*?-->` is a lazy run
+# in front of a REQUIRED literal: crafted input that repeats the opener and never
+# supplies `-->` makes every start position rescan the tail. Measured 20.1s at
+# 100_000 chars, exponent 1.96-2.14 over four doublings. Found by
+# docs/redos-sweep.py once it was generalised past the lexicon table.
+_REDOS_N = 100_000
+
+
+def test_crafted_comment_payload_stays_bounded():
+    payload = ("<!--" * (_REDOS_N // 4 + 1))[:_REDOS_N]
+    start = time.monotonic()
+    sanitize(payload)
+    assert time.monotonic() - start < 2.0
+
+
+def test_legitimate_comment_heavy_document_is_far_under_the_bound():
+    # The bound above only has signal if ordinary comment-dense content is
+    # nowhere near it: this is the same size, 100% closed comments.
+    unit = "<!-- a note -->"
+    payload = (unit * (_REDOS_N // len(unit) + 1))[:_REDOS_N]
+    start = time.monotonic()
+    sanitize(payload)
+    assert time.monotonic() - start < 0.5
+
+
+def test_comment_stripping_survives_the_redos_fix():
+    # Recall parity for every comment shape the lazy regex used to handle:
+    # nested markup, newlines (the pattern was DOTALL), and an unterminated
+    # comment, which must be left alone rather than swallowed to end-of-input.
+    assert sanitize("a <!-- <b>x</b> --> z").text == "a  z"
+    assert sanitize("a <!-- one\ntwo --> z").text == "a  z"
+    assert sanitize("a <!-- x --> b <!-- y --> c").text == "a  b  c"
+    assert sanitize("a <!-- never closed").text == "a <!-- never closed"
+    assert sanitize("a --> b").text == "a --> b"