1
0
Fork 0

fix(sanitize,okf,active_content): three quadratic patterns, two on the input path

The generalised sweep found what 0.3.2's hand-written rows missed. All three are
the documented class -- a run in front of a required literal that never arrives,
so every start position rescans the tail -- and all three are worse than the
0.3.3 findings, because `sanitize`, `neutralize`, `scan_active_content` and the
okf link graph apply NO input cap. `scan_lexicon`/`scan_output` are the only
entry points that do, so there is no ceiling to extrapolate to.

  sanitize._HTML_COMMENT_RE   `<!--`*100_000        20.1s, exponent 1.96-2.14
  active_content.URL_IN_TEXT_RE  `<a `+`A`*100_000  12.99s / 14.9s, exponent ~2.0
  okf._MD_LINK_RE             `[`*100_000            7.1s, exponent 1.99-2.05

Each fix is the one the pattern's own shape allows, not a copied choice:

  - The comment stripper drops the regex for `str.find`. Excluding `<` would lose
    every comment containing markup; bounding the run would be a carrier bypass
    of the exact construct the stripper exists to remove.
  - `URL_IN_TEXT_RE` bounds its scheme run to an RFC 3986 scheme (`{0,63}`).
    Bounding is safe *here* only because it is a defanger inside a tag already
    flagged `active:raw-html`. A lookbehind was measured too and rejected: it
    drops `-http://evil.com`, a one-character evasion. Bounded: 0.185s at 1M.
  - `_MD_LINK_RE` excludes `[`, matching `active_content.MD_LINK_RE` exactly,
    including the nested-label trade already documented there.

`sanitize` claimed "no catastrophic backtracking" in a comment; that claim was
wrong in the same way `output`'s was before 0.3.2, and is corrected in place.

676 tests (+10), coverage 128/128 + 6/6 gaps, sweep clean across 150 patterns.
The okf destination run gets no row: `[^)\s]+` cannot fail, so a row for it
could never go red.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-01 20:06:36 +02:00
commit 73fa1b99ae
9 changed files with 223 additions and 7 deletions

View file

@ -17,6 +17,8 @@ OKF spec facts used here (verified against okf/SPEC.md, 2026-07-06):
"""
import pytest
import time
from llm_ingestion_guard.okf import (
parse_frontmatter,
scan_concept,
@ -640,3 +642,34 @@ def test_t2_constrains_import_not_emission(fm):
doc = f"---\nid: x\n{fm}---\n\nbody\n"
assert import_bundle({"concepts/x.md": doc}).disposition is Disposition.FAIL_SECURE
assert screen_output(doc, PRESET_USER_UPLOAD).disposition is Disposition.WARN
# --- self-safety (OWASP LLM10): ReDoS in the link-graph extractor ------------
# `[^\]]*` is a run in front of a REQUIRED `]`: a bundle body that repeats `[`
# and never closes it makes every start position rescan the tail. Measured 7.1s
# at 100_000 chars, exponent 1.99-2.05 over four doublings, and the link graph
# runs over attacker-supplied bundle bodies with no input cap. Found by
# docs/redos-sweep.py once it was generalised past the lexicon table; the same
# defect in `active_content`'s markdown table was already fixed there the same
# way, by excluding the character that opens the pattern's own anchor.
_LINK_REDOS_N = 100_000
def test_crafted_link_payload_stays_bounded():
start = time.monotonic()
extract_link_targets("[" * _LINK_REDOS_N)
assert time.monotonic() - start < 2.0
# The destination run behind the label gets no row: `[^)\s]+` needs only one
# character, so it cannot fail, and a run that cannot fail cannot pay the
# per-start rescan. A row for it could never go red — decoration, not a pin.
def test_link_extraction_survives_the_redos_fix():
# Recall parity: ordinary links, a label holding brackets it does not close,
# and the nested-bracket form the exclusion deliberately gives up on -- the
# same trade `active_content.MD_LINK_RE` already makes.
assert extract_link_targets("see [x](./a.md) and [y](/b.md)") == ["./a.md", "/b.md"]
assert extract_link_targets("[a b](./c.md)") == ["./c.md"]
assert extract_link_targets("text [![img](./i.png)](./t.md)") == ["./i.png"]