1
0
Fork 0

release(0.3.2): the ReDoS fix, with both residuals measured against the v0.3.1 tag

Version sync in all four places (pyproject, __version__, README badge, README
install pin) plus the changelog entry for cff0437.

Two claims that were about to ship in the release note did not survive being
measured against a v0.3.1 worktree, and are corrected in LIMITATIONS first:

- "Report-only, so the cost is a review, not a block" was wrong twice. HIGH under
  a low-trust preset is fail_secure, not a review -- report-only means the text is
  never mutated, not that a finding cannot block. And the new script-tag false
  positive costs no consumer a disposition at all: any text containing a literal
  `<script>` already produced active:raw-html at HIGH on 0.3.1, so the same prose
  disposed fail_secure under PRESET_USER_UPLOAD before this change and after it.
  The fail-open that was closed is narrower for the same reason -- it existed only
  in scan_lexicon called on its own; through either composed gate, raw-html
  already caught the unclosed tag. What changed is the label, not the outcome.

- "The realistic long value is still caught by egress:jwt-token" was true and
  hid the part that matters. Measured at the 257-char boundary: a generic long
  password still trips entropy:base64-blob at CRITICAL, disposition unchanged.
  A JWT used as a DB password is the case that moves -- its remaining detections
  top out below CRITICAL, so the any-tier block is lost and PRESET_TRUSTED_SOURCE
  drops from fail_secure to quarantine_review. PRESET_USER_UPLOAD still
  fail_secures. Recorded as a behaviour change in the changelog, not buried.

The first probe for that boundary used a 300-char run of "A" and found nothing on
either version: all-same-char values are suppressed as placeholders. The probe was
wrong, not the pattern.

662 passed, coverage matrix exit 0, LIMITATIONS still 29 items = README's 29.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 21:16:45 +02:00
commit 75ae48277b
5 changed files with 99 additions and 9 deletions

View file

@ -7,6 +7,81 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.3.2] — 2026-07-31
> **Denial-of-service fix. Upgrade from 0.3.1.** The output gate could be made to
> spend hours on a single call by crafted input it accepts by design. No
> disposition changes for ordinary documents — the one measured exception is
> listed under *Known behaviour changes* below. The v0.3.1 tag is not moved.
### Fixed — 19 quadratic regex runs on the output path
`scan_output` claimed LLM10 self-safety on the grounds that its patterns contain no
nested quantifiers. That is true and it is not the property that matters. A run in
front of a **required** literal, reachable from a short anchor, is enough: crafted
input repeats the anchor and never supplies the literal, so every start position
rescans the tail. Quadratic, not exponential — and quadratic is sufficient here.
Measured, not argued (Python 3.14, this machine):
| Input | Time through `scan_output` |
|---|---|
| `<a:` × 100 000 (300 KB) | **458.7 s** |
| size-matched ordinary prose | 0.31 s |
| same payload extrapolated to the 1 000 000-char input the gate itself accepts | **~5.7 hours for one call** |
`max_scan_chars` does not mitigate this. It bounds the *input*; quadratic work on a
bounded input is still hours. That claim was stated in both `output.py` and
`calibration.py` and is corrected in both.
The fix is per pattern, not uniform:
- **`active_content` + the lexicon table (15 runs)** — exclude the character that
opens the pattern's own anchor (`[` for markdown, `<` for tags), so a run cannot
reach past the next start position and the per-start costs telescope. Verified to
cost no recall: long URLs, long alt text, and `<` inside a quoted attribute all
still match. Bounding instead would have been linear too, but wrong here — the
content is attacker-controlled, so padding past a bound would be a one-line bypass
of the EchoLeak class this table exists to catch.
- **`*-connstr` secret egress (4 runs)** — bound the password at the new
`MAX_CONNSTR_VALUE` (256) in `calibration`. The exclusion fix is unavailable: the
anchor character is `/`, and passwords containing `/` are the common case
(measured — they match today).
- **`hybrid-xss:script-tag`** — had neither option, since its run is the script
*body*, which may legitimately contain `<`. It now matches the opening tag and no
longer requires `</script>`.
### Known behaviour changes
Two, both measured against the v0.3.1 tag rather than reasoned about:
- **A JWT used as a DB password, over 256 chars, is no longer CRITICAL.** The
remaining detections (`entropy:base64-blob` HIGH, `egress:jwt-token` MEDIUM) top
out below CRITICAL, so the any-tier block is lost: under `PRESET_TRUSTED_SOURCE`
such a document moves from `fail_secure` to `quarantine_review`. Under
`PRESET_USER_UPLOAD` it still `fail_secure`s, and a *generic* long password still
trips `entropy:base64-blob` at CRITICAL with no change at all. The credential is
never silently missed; on one preset it is held for review instead of halted.
- **`hybrid-xss:script-tag` now fires on prose that merely mentions `<script>`** —
and this costs no consumer a disposition. Any text containing a literal
`<script>` already produced `active:raw-html` at HIGH on 0.3.1, so the same
document disposed identically before and after. The label is new; the outcome is
not. By the same measurement, the fail-open this closed (an unclosed
`<script>alert(1)`) was confined to `scan_lexicon` called on its own — through
either composed gate, `active:raw-html` already caught it.
Both are recorded in `docs/LIMITATIONS.md` (still 29 items — these replace nothing).
### Method note
The defect was found by a composed-gate DoS test that stayed red after every
individual scanner had been made linear; the remaining 813× was the lexicon's six
`html-obfuscation` patterns. A per-scanner test alone would have shipped it. The
static shape analysis used to find candidates also missed `[\s\S]*?` in
`script-tag` — the sweep that matters is measurement, not a regex over regexes.
662 tests pass (was 642), and the suite is faster than before the fix.
## [0.3.1] — 2026-07-25
> **Regression fix. Upgrade from 0.3.0.** v0.3.0 made the high-untrust upload path

View file

@ -1,6 +1,6 @@
# llm-ingestion-guard
![Version](https://img.shields.io/badge/version-0.3.1-blue)
![Version](https://img.shields.io/badge/version-0.3.2-blue)
![Status](https://img.shields.io/badge/status-alpha-orange)
![Python](https://img.shields.io/badge/python-3.10%2B-purple)
![Tests](https://img.shields.io/badge/tests-642_passing-green)
@ -42,7 +42,7 @@ may still change. There are real limitations, stated plainly below; read them.
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
```bash
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.1"
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.2"
```
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or

View file

@ -248,17 +248,32 @@ items; this is the full list, each with the mechanism.
pattern matches the opening tag and no longer requires `</script>`, so a
document *about* XSS is flagged alongside a document that *carries* it. This
is a deliberate trade made twice over: requiring the closing tag was a
fail-open (an unclosed `<script>alert(1)` was silently missed) and it was the
last quadratic-backtracking site on the output path. Report-only, so the cost
is a review, not a block.
fail-open (an unclosed `<script>alert(1)` was silently missed by *this label*)
and it was the last quadratic-backtracking site on the output path.
**Measured, both claims are narrower than they read.** The new label costs no
consumer a disposition: any text containing a literal `<script>` already
produced `active:raw-html` at HIGH on 0.3.1 — so the same prose disposed
`fail_secure` under `PRESET_USER_UPLOAD` before this change and after it. The
fail-open was equally confined to `scan_lexicon` called on its own; through
either composed gate, `active:raw-html` already caught the unclosed tag. What
changed is the *label*, not the outcome. And the outcome is not "a review":
HIGH under a low-trust preset is `fail_secure`. Report-only means the text is
never mutated — it does not mean the finding cannot block.
- **A connection-string password longer than 256 chars is not matched.** The
password run in the `*-connstr` egress patterns is bounded by
`MAX_CONNSTR_VALUE`; unbounded, it sits in front of a mandatory `@` and makes
crafted input quadratic. Excluding the anchor character instead — the fix the
active-content table uses — is unavailable here because that character is `/`,
and a password containing `/` is the common case. The realistic long value (a
token used as a DB password) is still caught by `egress:jwt-token`.
and a password containing `/` is the common case. **What the residual actually
costs, measured at the 257-char boundary:** a generic long password still trips
`entropy:base64-blob` at CRITICAL, so the disposition is unchanged. A *JWT* used
as a DB password is the case that moves — the remaining detections
(`entropy:base64-blob` HIGH, `egress:jwt-token` MEDIUM) top out below CRITICAL,
so the any-tier CRITICAL block is lost: under `PRESET_TRUSTED_SOURCE` such a
document drops from `fail_secure` to `quarantine_review`. Under
`PRESET_USER_UPLOAD` it still `fail_secure`s. The credential is never silently
missed; on one preset it is held for review instead of halted.
## The six documented gaps (tracked by the coverage matrix)

View file

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "llm-ingestion-guard"
version = "0.3.1"
version = "0.3.2"
description = "A minimal, dependency-light defensive layer for LLM ingestion pipelines — the write-time siblings of query-time chatbot guardrails."
readme = "README.md"
requires-python = ">=3.10"

View file

@ -59,7 +59,7 @@ from .grounding import (
)
from . import okf
__version__ = "0.3.1"
__version__ = "0.3.2"
# --- §6 bookends: the two library-side halves around the transform ---------