release(0.3.2): the ReDoS fix, with both residuals measured against the v0.3.1 tag
Version sync in all four places (pyproject, __version__, README badge, README
install pin) plus the changelog entry for cff0437.
Two claims that were about to ship in the release note did not survive being
measured against a v0.3.1 worktree, and are corrected in LIMITATIONS first:
- "Report-only, so the cost is a review, not a block" was wrong twice. HIGH under
a low-trust preset is fail_secure, not a review -- report-only means the text is
never mutated, not that a finding cannot block. And the new script-tag false
positive costs no consumer a disposition at all: any text containing a literal
`<script>` already produced active:raw-html at HIGH on 0.3.1, so the same prose
disposed fail_secure under PRESET_USER_UPLOAD before this change and after it.
The fail-open that was closed is narrower for the same reason -- it existed only
in scan_lexicon called on its own; through either composed gate, raw-html
already caught the unclosed tag. What changed is the label, not the outcome.
- "The realistic long value is still caught by egress:jwt-token" was true and
hid the part that matters. Measured at the 257-char boundary: a generic long
password still trips entropy:base64-blob at CRITICAL, disposition unchanged.
A JWT used as a DB password is the case that moves -- its remaining detections
top out below CRITICAL, so the any-tier block is lost and PRESET_TRUSTED_SOURCE
drops from fail_secure to quarantine_review. PRESET_USER_UPLOAD still
fail_secures. Recorded as a behaviour change in the changelog, not buried.
The first probe for that boundary used a 300-char run of "A" and found nothing on
either version: all-same-char values are suppressed as placeholders. The probe was
wrong, not the pattern.
662 passed, coverage matrix exit 0, LIMITATIONS still 29 items = README's 29.
This commit is contained in:
parent
cff043787d
commit
75ae48277b
5 changed files with 99 additions and 9 deletions
75
CHANGELOG.md
75
CHANGELOG.md
|
|
@ -7,6 +7,81 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.3.2] — 2026-07-31
|
||||
|
||||
> **Denial-of-service fix. Upgrade from 0.3.1.** The output gate could be made to
|
||||
> spend hours on a single call by crafted input it accepts by design. No
|
||||
> disposition changes for ordinary documents — the one measured exception is
|
||||
> listed under *Known behaviour changes* below. The v0.3.1 tag is not moved.
|
||||
|
||||
### Fixed — 19 quadratic regex runs on the output path
|
||||
|
||||
`scan_output` claimed LLM10 self-safety on the grounds that its patterns contain no
|
||||
nested quantifiers. That is true and it is not the property that matters. A run in
|
||||
front of a **required** literal, reachable from a short anchor, is enough: crafted
|
||||
input repeats the anchor and never supplies the literal, so every start position
|
||||
rescans the tail. Quadratic, not exponential — and quadratic is sufficient here.
|
||||
|
||||
Measured, not argued (Python 3.14, this machine):
|
||||
|
||||
| Input | Time through `scan_output` |
|
||||
|---|---|
|
||||
| `<a:` × 100 000 (300 KB) | **458.7 s** |
|
||||
| size-matched ordinary prose | 0.31 s |
|
||||
| same payload extrapolated to the 1 000 000-char input the gate itself accepts | **~5.7 hours for one call** |
|
||||
|
||||
`max_scan_chars` does not mitigate this. It bounds the *input*; quadratic work on a
|
||||
bounded input is still hours. That claim was stated in both `output.py` and
|
||||
`calibration.py` and is corrected in both.
|
||||
|
||||
The fix is per pattern, not uniform:
|
||||
|
||||
- **`active_content` + the lexicon table (15 runs)** — exclude the character that
|
||||
opens the pattern's own anchor (`[` for markdown, `<` for tags), so a run cannot
|
||||
reach past the next start position and the per-start costs telescope. Verified to
|
||||
cost no recall: long URLs, long alt text, and `<` inside a quoted attribute all
|
||||
still match. Bounding instead would have been linear too, but wrong here — the
|
||||
content is attacker-controlled, so padding past a bound would be a one-line bypass
|
||||
of the EchoLeak class this table exists to catch.
|
||||
- **`*-connstr` secret egress (4 runs)** — bound the password at the new
|
||||
`MAX_CONNSTR_VALUE` (256) in `calibration`. The exclusion fix is unavailable: the
|
||||
anchor character is `/`, and passwords containing `/` are the common case
|
||||
(measured — they match today).
|
||||
- **`hybrid-xss:script-tag`** — had neither option, since its run is the script
|
||||
*body*, which may legitimately contain `<`. It now matches the opening tag and no
|
||||
longer requires `</script>`.
|
||||
|
||||
### Known behaviour changes
|
||||
|
||||
Two, both measured against the v0.3.1 tag rather than reasoned about:
|
||||
|
||||
- **A JWT used as a DB password, over 256 chars, is no longer CRITICAL.** The
|
||||
remaining detections (`entropy:base64-blob` HIGH, `egress:jwt-token` MEDIUM) top
|
||||
out below CRITICAL, so the any-tier block is lost: under `PRESET_TRUSTED_SOURCE`
|
||||
such a document moves from `fail_secure` to `quarantine_review`. Under
|
||||
`PRESET_USER_UPLOAD` it still `fail_secure`s, and a *generic* long password still
|
||||
trips `entropy:base64-blob` at CRITICAL with no change at all. The credential is
|
||||
never silently missed; on one preset it is held for review instead of halted.
|
||||
- **`hybrid-xss:script-tag` now fires on prose that merely mentions `<script>`** —
|
||||
and this costs no consumer a disposition. Any text containing a literal
|
||||
`<script>` already produced `active:raw-html` at HIGH on 0.3.1, so the same
|
||||
document disposed identically before and after. The label is new; the outcome is
|
||||
not. By the same measurement, the fail-open this closed (an unclosed
|
||||
`<script>alert(1)`) was confined to `scan_lexicon` called on its own — through
|
||||
either composed gate, `active:raw-html` already caught it.
|
||||
|
||||
Both are recorded in `docs/LIMITATIONS.md` (still 29 items — these replace nothing).
|
||||
|
||||
### Method note
|
||||
|
||||
The defect was found by a composed-gate DoS test that stayed red after every
|
||||
individual scanner had been made linear; the remaining 813× was the lexicon's six
|
||||
`html-obfuscation` patterns. A per-scanner test alone would have shipped it. The
|
||||
static shape analysis used to find candidates also missed `[\s\S]*?` in
|
||||
`script-tag` — the sweep that matters is measurement, not a regex over regexes.
|
||||
|
||||
662 tests pass (was 642), and the suite is faster than before the fix.
|
||||
|
||||
## [0.3.1] — 2026-07-25
|
||||
|
||||
> **Regression fix. Upgrade from 0.3.0.** v0.3.0 made the high-untrust upload path
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# llm-ingestion-guard
|
||||
|
||||

|
||||

|
||||

|
||||

|
||||

|
||||
|
|
@ -42,7 +42,7 @@ may still change. There are real limitations, stated plainly below; read them.
|
|||
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
|
||||
|
||||
```bash
|
||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.1"
|
||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.2"
|
||||
```
|
||||
|
||||
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
|
||||
|
|
|
|||
|
|
@ -248,17 +248,32 @@ items; this is the full list, each with the mechanism.
|
|||
pattern matches the opening tag and no longer requires `</script>`, so a
|
||||
document *about* XSS is flagged alongside a document that *carries* it. This
|
||||
is a deliberate trade made twice over: requiring the closing tag was a
|
||||
fail-open (an unclosed `<script>alert(1)` was silently missed) and it was the
|
||||
last quadratic-backtracking site on the output path. Report-only, so the cost
|
||||
is a review, not a block.
|
||||
fail-open (an unclosed `<script>alert(1)` was silently missed by *this label*)
|
||||
and it was the last quadratic-backtracking site on the output path.
|
||||
**Measured, both claims are narrower than they read.** The new label costs no
|
||||
consumer a disposition: any text containing a literal `<script>` already
|
||||
produced `active:raw-html` at HIGH on 0.3.1 — so the same prose disposed
|
||||
`fail_secure` under `PRESET_USER_UPLOAD` before this change and after it. The
|
||||
fail-open was equally confined to `scan_lexicon` called on its own; through
|
||||
either composed gate, `active:raw-html` already caught the unclosed tag. What
|
||||
changed is the *label*, not the outcome. And the outcome is not "a review":
|
||||
HIGH under a low-trust preset is `fail_secure`. Report-only means the text is
|
||||
never mutated — it does not mean the finding cannot block.
|
||||
|
||||
- **A connection-string password longer than 256 chars is not matched.** The
|
||||
password run in the `*-connstr` egress patterns is bounded by
|
||||
`MAX_CONNSTR_VALUE`; unbounded, it sits in front of a mandatory `@` and makes
|
||||
crafted input quadratic. Excluding the anchor character instead — the fix the
|
||||
active-content table uses — is unavailable here because that character is `/`,
|
||||
and a password containing `/` is the common case. The realistic long value (a
|
||||
token used as a DB password) is still caught by `egress:jwt-token`.
|
||||
and a password containing `/` is the common case. **What the residual actually
|
||||
costs, measured at the 257-char boundary:** a generic long password still trips
|
||||
`entropy:base64-blob` at CRITICAL, so the disposition is unchanged. A *JWT* used
|
||||
as a DB password is the case that moves — the remaining detections
|
||||
(`entropy:base64-blob` HIGH, `egress:jwt-token` MEDIUM) top out below CRITICAL,
|
||||
so the any-tier CRITICAL block is lost: under `PRESET_TRUSTED_SOURCE` such a
|
||||
document drops from `fail_secure` to `quarantine_review`. Under
|
||||
`PRESET_USER_UPLOAD` it still `fail_secure`s. The credential is never silently
|
||||
missed; on one preset it is held for review instead of halted.
|
||||
|
||||
## The six documented gaps (tracked by the coverage matrix)
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "llm-ingestion-guard"
|
||||
version = "0.3.1"
|
||||
version = "0.3.2"
|
||||
description = "A minimal, dependency-light defensive layer for LLM ingestion pipelines — the write-time siblings of query-time chatbot guardrails."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
|
|
|
|||
|
|
@ -59,7 +59,7 @@ from .grounding import (
|
|||
)
|
||||
from . import okf
|
||||
|
||||
__version__ = "0.3.1"
|
||||
__version__ = "0.3.2"
|
||||
|
||||
|
||||
# --- §6 bookends: the two library-side halves around the transform ---------
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue