release(0.3.4): three quadratic patterns outside the lexicon, two on the input path

0.3.3 swept 83 lexicon patterns arm by arm and left the other ten regex-bearing
modules on 0.3.2's hand-written rows. Generalising the sweep found three more,
and the two on the input path matter more than the count suggests: `sanitize` is
step 1 of `prepare_input`, and `MAX_SCAN_CHARS` is applied in `scan_lexicon` and
`scan_output` only, so there was no cap to extrapolate to. That missing input
cap is now a documented residual of its own -- extending it changes the contract
for existing callers and is not something to smuggle into a ReDoS patch.

Version synced in all four places + CHANGELOG. LIMITATIONS 30 -> 31 items;
the 0.3.3 entry claiming the lexicon sweep's scope is corrected in place, since
"the output path" was never the whole surface either.

676 tests, coverage 128/128 + 6/6 gaps, sweep clean across 150 patterns.

CHANGELOG.md

@@ -7,6 +7,76 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.3.4] — 2026-08-01
+
+> **Denial-of-service fix on the INPUT path. Upgrade from 0.3.3.** 0.3.3 swept
+> the 83 lexicon patterns arm by arm and left every other table on 0.3.2's
+> hand-written rows. Generalising the sweep over all eleven regex-bearing modules
+> found three more quadratic patterns — two of them on the input path, one in
+> `sanitize`, the first thing every ingested document touches. No disposition
+> changes: recall was measured case by case and nothing was lost. Earlier tags
+> are not moved.
+
+### Fixed — three quadratic patterns, two on the input path
+
+Same class as everything 0.3.2 and 0.3.3 fixed: a run in front of a **required**
+literal, so crafted input that never supplies the literal makes every start
+position rescan the tail. Each exponent is read across four doublings, not from a
+two-point ratio.
+
+| Pattern | Crafted payload | Measured @ 100 000 | Exponent |
+|---|---|---|---|
+| `sanitize._HTML_COMMENT_RE` | `<!--` × N | **20.1 s** | 1.96–2.14 |
+| `active_content.URL_IN_TEXT_RE` | `<a ` + `A` × N + `>` | 12.99 s / 14.9 s | 1.87–2.22 |
+| `okf._MD_LINK_RE` | `[` × N | 7.1 s | 1.99–2.05 |
+
+These are worse than the 0.3.3 findings, and the reason is a separate finding of
+its own: `MAX_SCAN_CHARS` is applied in `scan_lexicon` and `scan_output` **only**.
+`sanitize`, `neutralize`, `scan_active_content` and the okf link graph accept
+input of any size, so there is no cap to extrapolate to. Now documented as a
+residual in `docs/LIMITATIONS.md`; extending the cap into the input path changes
+the contract for existing callers and is deliberately not done in a ReDoS patch.
+
+Each fix is the one the pattern's own shape allows — the 0.3.3 lesson that a fix
+choice must not be copied blindly from a neighbouring table:
+
+- **`sanitize`** drops the regex for a `str.find` scan, semantically identical to
+  the lazy `<!--.*?-->` it replaces. Excluding `<` from the run would lose every
+  comment containing markup (`<!-- <b>x</b> -->` is the ordinary case); bounding
+  the run would be a one-line carrier bypass of the exact construct the stripper
+  exists to remove. The module's own "no catastrophic backtracking" comment was
+  wrong in the same way `output`'s was before 0.3.2, and is corrected in place.
+- **`URL_IN_TEXT_RE`** bounds its scheme run to an RFC 3986 scheme (`{0,63}`).
+  Bounding is safe *here* only because this is a defanger applied inside a tag
+  already flagged `active:raw-html`, so padding shifts where the match starts
+  rather than evading detection. A lookbehind killing interior start positions
+  was measured too and **rejected**: it drops `-http://evil.com` and
+  `.http://x.com`, a one-character evasion of the defanger. Bounded, the pattern
+  runs in 0.185 s at the full 1 000 000-char cap.
+- **`okf._MD_LINK_RE`** excludes `[`, matching `active_content.MD_LINK_RE`
+  exactly, including the nested-label trade already documented there.
+
+### Changed — the sweep covers every regex surface, not one table
+
+`docs/redos-sweep.py` now sweeps **150 patterns across 11 tables** (0.3.3 covered
+83 in one). The collector is mechanical on both axes so no one has to remember to
+list anything: it walks each module's namespace for compiled patterns, and it
+derives each pattern's call mode from the module source, because `.sub()` and
+`.finditer()` visit every start position where `.match()` cannot. A pattern
+reachable only through a helper parameter gets the worst mode, marked `*` — the
+fallback can over-measure but never miss.
+
+Two arm shapes the generator cannot express are pinned by hand as a result: a tag
+that *closes* around a long body (repeating-unit payloads never close it), and a
+run of plain characters carrying no anchor at all. The `okf` destination run gets
+no row on purpose: `[^)\s]+` cannot fail, so a pin for it could never go red.
+
+A lexicon candidate flagged at ×2.8 measured **linear** across four doublings
+(exponent 0.96–1.03) — the near-noise-floor false flag the script's own docstring
+warns about, confirmed a second time.
+
+676 tests (+10). Coverage matrix unchanged at 128/128 caught, 6/6 gaps holding.
+
 ## [0.3.3] — 2026-07-31
 
 > **Denial-of-service fix, and a correction to 0.3.2. Upgrade from 0.3.2.** The

README.md

@@ -1,9 +1,9 @@
 # llm-ingestion-guard
 
-![Version](https://img.shields.io/badge/version-0.3.3-blue)
+![Version](https://img.shields.io/badge/version-0.3.4-blue)
 ![Status](https://img.shields.io/badge/status-alpha-orange)
 ![Python](https://img.shields.io/badge/python-3.10%2B-purple)
-![Tests](https://img.shields.io/badge/tests-666_passing-green)
+![Tests](https://img.shields.io/badge/tests-676_passing-green)
 ![License](https://img.shields.io/badge/license-MIT-lightgrey)
 
 **Write-time ingestion is the trust boundary that query-time guardrails
@@ -42,7 +42,7 @@ may still change. There are real limitations, stated plainly below; read them.
 Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
 
 ```bash
-pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.3"
+pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.4"
 ```
 
 The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
@@ -215,7 +215,7 @@ a green scan means safe content. The highest-impact items:
   egress, semantic poisoning, trusted-prose lone-HIGH, lexicon dedup (`count=1`),
   pure beaconing, and short opaque URL segments.
 
-**Full list — 30 items, each with the mechanism, plus the out-of-scope boundary:**
+**Full list — 31 items, each with the mechanism, plus the out-of-scope boundary:**
 [`docs/LIMITATIONS.md`](docs/LIMITATIONS.md). Several carry field measurements from
 consumer corpora, including the false positives the URL-shape rule actually produces.
 

docs/LIMITATIONS.md

@@ -255,6 +255,9 @@ items; this is the full list, each with the mechanism.
   output path too, they were reachable through `scan_output`: `"[" * 100_000`
   took 334.7s through the gate. The claim was too broad because the sweep behind
   it drove `[` only through `scan_active_content`, never through the lexicon.
+  0.3.4 then found three more outside the lexicon — two of them on the *input*
+  path, where no cap applies at all — so "the output path" was never the whole
+  surface either.
   **Measured, both claims are narrower than they read.** The new label costs no
   consumer a disposition: any text containing a literal `<script>` already
   produced `active:raw-html` at HIGH on 0.3.1 — so the same prose disposed
@@ -280,18 +283,38 @@ items; this is the full list, each with the mechanism.
   `PRESET_USER_UPLOAD` it still `fail_secure`s. The credential is never silently
   missed; on one preset it is held for review instead of halted.
 
-- **The ReDoS sweep of the lexicon has a measured sensitivity floor, not a clean
-  bill of health.** All 83 patterns were swept arm by arm — payloads synthesised
-  per run from each pattern's own skeleton, so `[`, `[system]` and `[system](`
-  are each probed separately rather than relying on generic units. Two patterns
-  were quadratic and both are fixed. But the sweep flags on *timing*, and it
-  ignores measurements below a 1.5 ms noise floor at N=8000. A quadratic arm
-  sitting just under that floor would still cost **up to ~23 s** at the
-  1 000 000-char cap the gate accepts. So the claim this sweep supports is
-  "no arm worse than ~23 s at the cap", not "no quadratic arm remains". The
-  method's blind spot is real and was demonstrated in this very sweep: a
-  generic-payload pass found only one of the two patterns, and the second
-  surfaced only after the payloads were generated per run.
+- **The ReDoS sweep has a measured sensitivity floor, not a clean bill of
+  health.** All 150 compiled patterns across all eleven regex-bearing modules are
+  swept arm by arm — payloads synthesised per run from each pattern's own
+  skeleton, so `[`, `[system]` and `[system](` are each probed separately rather
+  than relying on generic units, and each pattern is timed in the call mode the
+  production code uses (`.sub()`/`.finditer()` visit every start position where
+  `.match()` cannot). Five patterns were quadratic across 0.3.3 and 0.3.4; all
+  are fixed. But the sweep flags on *timing*, and it ignores measurements below a
+  1.5 ms noise floor at N=8000. A quadratic arm sitting just under that floor
+  would still cost **up to ~23 s** at the 1 000 000-char cap. So the claim this
+  sweep supports is "no arm worse than ~23 s at the cap", not "no quadratic arm
+  remains". The method's blind spot is real and has now been demonstrated twice:
+  a generic-payload pass found only one of 0.3.3's two patterns, and 0.3.2's
+  hand-written rows missed all three of 0.3.4's — including one on `sanitize`,
+  the first thing every ingested document touches. **Two arm shapes the unit-
+  repetition payloads cannot express** are pinned by hand as a result: a tag that
+  *closes* around a long body, and a run of plain characters carrying no anchor
+  at all.
+
+- **Only `scan_lexicon` and `scan_output` cap their input; the input-side entry
+  points do not.** `MAX_SCAN_CHARS` (1 000 000) is applied in those two functions
+  only. `sanitize`, `fence`, `neutralize`, `scan_active_content` and the okf link
+  graph accept text of any length, so their cost is bounded by the caller's
+  input, not by this library. Every *known* quadratic run on those paths is
+  fixed, and the residual above states what the sweep can and cannot claim — but
+  where an output-path residual is capped at ~23 s, the same residual on the
+  input path has no ceiling. A caller that ingests untrusted documents of
+  unbounded size should impose its own limit before `prepare_input`. Extending
+  the cap into the input path is deliberately **not** done as part of a ReDoS
+  patch: it changes the contract for existing callers (what happens to the
+  truncated remainder is a policy question), and that deserves its own decision
+  rather than being smuggled in.
 
 ## The six documented gaps (tracked by the coverage matrix)
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "llm-ingestion-guard"
-version = "0.3.3"
+version = "0.3.4"
 description = "A minimal, dependency-light defensive layer for LLM ingestion pipelines — the write-time siblings of query-time chatbot guardrails."
 readme = "README.md"
 requires-python = ">=3.10"

src/llm_ingestion_guard/__init__.py

@@ -59,7 +59,7 @@ from .grounding import (
 )
 from . import okf
 
-__version__ = "0.3.3"
+__version__ = "0.3.4"
 
 
 # --- §6 bookends: the two library-side halves around the transform ---------