release(0.3.2): the ReDoS fix, with both residuals measured against the v0.3.1 tag
Version sync in all four places (pyproject, __version__, README badge, README
install pin) plus the changelog entry for cff0437.
Two claims that were about to ship in the release note did not survive being
measured against a v0.3.1 worktree, and are corrected in LIMITATIONS first:
- "Report-only, so the cost is a review, not a block" was wrong twice. HIGH under
a low-trust preset is fail_secure, not a review -- report-only means the text is
never mutated, not that a finding cannot block. And the new script-tag false
positive costs no consumer a disposition at all: any text containing a literal
`<script>` already produced active:raw-html at HIGH on 0.3.1, so the same prose
disposed fail_secure under PRESET_USER_UPLOAD before this change and after it.
The fail-open that was closed is narrower for the same reason -- it existed only
in scan_lexicon called on its own; through either composed gate, raw-html
already caught the unclosed tag. What changed is the label, not the outcome.
- "The realistic long value is still caught by egress:jwt-token" was true and
hid the part that matters. Measured at the 257-char boundary: a generic long
password still trips entropy:base64-blob at CRITICAL, disposition unchanged.
A JWT used as a DB password is the case that moves -- its remaining detections
top out below CRITICAL, so the any-tier block is lost and PRESET_TRUSTED_SOURCE
drops from fail_secure to quarantine_review. PRESET_USER_UPLOAD still
fail_secures. Recorded as a behaviour change in the changelog, not buried.
The first probe for that boundary used a 300-char run of "A" and found nothing on
either version: all-same-char values are suppressed as placeholders. The probe was
wrong, not the pattern.
662 passed, coverage matrix exit 0, LIMITATIONS still 29 items = README's 29.
This commit is contained in:
parent
cff043787d
commit
f245f230a8
5 changed files with 99 additions and 9 deletions
75
CHANGELOG.md
75
CHANGELOG.md
|
|
@ -7,6 +7,81 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
## [0.3.2] — 2026-07-31
|
||||||
|
|
||||||
|
> **Denial-of-service fix. Upgrade from 0.3.1.** The output gate could be made to
|
||||||
|
> spend hours on a single call by crafted input it accepts by design. No
|
||||||
|
> disposition changes for ordinary documents — the one measured exception is
|
||||||
|
> listed under *Known behaviour changes* below. The v0.3.1 tag is not moved.
|
||||||
|
|
||||||
|
### Fixed — 19 quadratic regex runs on the output path
|
||||||
|
|
||||||
|
`scan_output` claimed LLM10 self-safety on the grounds that its patterns contain no
|
||||||
|
nested quantifiers. That is true and it is not the property that matters. A run in
|
||||||
|
front of a **required** literal, reachable from a short anchor, is enough: crafted
|
||||||
|
input repeats the anchor and never supplies the literal, so every start position
|
||||||
|
rescans the tail. Quadratic, not exponential — and quadratic is sufficient here.
|
||||||
|
|
||||||
|
Measured, not argued (Python 3.14, this machine):
|
||||||
|
|
||||||
|
| Input | Time through `scan_output` |
|
||||||
|
|---|---|
|
||||||
|
| `<a:` × 100 000 (300 KB) | **458.7 s** |
|
||||||
|
| size-matched ordinary prose | 0.31 s |
|
||||||
|
| same payload extrapolated to the 1 000 000-char input the gate itself accepts | **~5.7 hours for one call** |
|
||||||
|
|
||||||
|
`max_scan_chars` does not mitigate this. It bounds the *input*; quadratic work on a
|
||||||
|
bounded input is still hours. That claim was stated in both `output.py` and
|
||||||
|
`calibration.py` and is corrected in both.
|
||||||
|
|
||||||
|
The fix is per pattern, not uniform:
|
||||||
|
|
||||||
|
- **`active_content` + the lexicon table (15 runs)** — exclude the character that
|
||||||
|
opens the pattern's own anchor (`[` for markdown, `<` for tags), so a run cannot
|
||||||
|
reach past the next start position and the per-start costs telescope. Verified to
|
||||||
|
cost no recall: long URLs, long alt text, and `<` inside a quoted attribute all
|
||||||
|
still match. Bounding instead would have been linear too, but wrong here — the
|
||||||
|
content is attacker-controlled, so padding past a bound would be a one-line bypass
|
||||||
|
of the EchoLeak class this table exists to catch.
|
||||||
|
- **`*-connstr` secret egress (4 runs)** — bound the password at the new
|
||||||
|
`MAX_CONNSTR_VALUE` (256) in `calibration`. The exclusion fix is unavailable: the
|
||||||
|
anchor character is `/`, and passwords containing `/` are the common case
|
||||||
|
(measured — they match today).
|
||||||
|
- **`hybrid-xss:script-tag`** — had neither option, since its run is the script
|
||||||
|
*body*, which may legitimately contain `<`. It now matches the opening tag and no
|
||||||
|
longer requires `</script>`.
|
||||||
|
|
||||||
|
### Known behaviour changes
|
||||||
|
|
||||||
|
Two, both measured against the v0.3.1 tag rather than reasoned about:
|
||||||
|
|
||||||
|
- **A JWT used as a DB password, over 256 chars, is no longer CRITICAL.** The
|
||||||
|
remaining detections (`entropy:base64-blob` HIGH, `egress:jwt-token` MEDIUM) top
|
||||||
|
out below CRITICAL, so the any-tier block is lost: under `PRESET_TRUSTED_SOURCE`
|
||||||
|
such a document moves from `fail_secure` to `quarantine_review`. Under
|
||||||
|
`PRESET_USER_UPLOAD` it still `fail_secure`s, and a *generic* long password still
|
||||||
|
trips `entropy:base64-blob` at CRITICAL with no change at all. The credential is
|
||||||
|
never silently missed; on one preset it is held for review instead of halted.
|
||||||
|
- **`hybrid-xss:script-tag` now fires on prose that merely mentions `<script>`** —
|
||||||
|
and this costs no consumer a disposition. Any text containing a literal
|
||||||
|
`<script>` already produced `active:raw-html` at HIGH on 0.3.1, so the same
|
||||||
|
document disposed identically before and after. The label is new; the outcome is
|
||||||
|
not. By the same measurement, the fail-open this closed (an unclosed
|
||||||
|
`<script>alert(1)`) was confined to `scan_lexicon` called on its own — through
|
||||||
|
either composed gate, `active:raw-html` already caught it.
|
||||||
|
|
||||||
|
Both are recorded in `docs/LIMITATIONS.md` (still 29 items — these replace nothing).
|
||||||
|
|
||||||
|
### Method note
|
||||||
|
|
||||||
|
The defect was found by a composed-gate DoS test that stayed red after every
|
||||||
|
individual scanner had been made linear; the remaining 813× was the lexicon's six
|
||||||
|
`html-obfuscation` patterns. A per-scanner test alone would have shipped it. The
|
||||||
|
static shape analysis used to find candidates also missed `[\s\S]*?` in
|
||||||
|
`script-tag` — the sweep that matters is measurement, not a regex over regexes.
|
||||||
|
|
||||||
|
662 tests pass (was 642), and the suite is faster than before the fix.
|
||||||
|
|
||||||
## [0.3.1] — 2026-07-25
|
## [0.3.1] — 2026-07-25
|
||||||
|
|
||||||
> **Regression fix. Upgrade from 0.3.0.** v0.3.0 made the high-untrust upload path
|
> **Regression fix. Upgrade from 0.3.0.** v0.3.0 made the high-untrust upload path
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
# llm-ingestion-guard
|
# llm-ingestion-guard
|
||||||
|
|
||||||

|

|
||||||

|

|
||||||

|

|
||||||

|

|
||||||
|
|
@ -42,7 +42,7 @@ may still change. There are real limitations, stated plainly below; read them.
|
||||||
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
|
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.1"
|
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.2"
|
||||||
```
|
```
|
||||||
|
|
||||||
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
|
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
|
||||||
|
|
|
||||||
|
|
@ -248,17 +248,32 @@ items; this is the full list, each with the mechanism.
|
||||||
pattern matches the opening tag and no longer requires `</script>`, so a
|
pattern matches the opening tag and no longer requires `</script>`, so a
|
||||||
document *about* XSS is flagged alongside a document that *carries* it. This
|
document *about* XSS is flagged alongside a document that *carries* it. This
|
||||||
is a deliberate trade made twice over: requiring the closing tag was a
|
is a deliberate trade made twice over: requiring the closing tag was a
|
||||||
fail-open (an unclosed `<script>alert(1)` was silently missed) and it was the
|
fail-open (an unclosed `<script>alert(1)` was silently missed by *this label*)
|
||||||
last quadratic-backtracking site on the output path. Report-only, so the cost
|
and it was the last quadratic-backtracking site on the output path.
|
||||||
is a review, not a block.
|
**Measured, both claims are narrower than they read.** The new label costs no
|
||||||
|
consumer a disposition: any text containing a literal `<script>` already
|
||||||
|
produced `active:raw-html` at HIGH on 0.3.1 — so the same prose disposed
|
||||||
|
`fail_secure` under `PRESET_USER_UPLOAD` before this change and after it. The
|
||||||
|
fail-open was equally confined to `scan_lexicon` called on its own; through
|
||||||
|
either composed gate, `active:raw-html` already caught the unclosed tag. What
|
||||||
|
changed is the *label*, not the outcome. And the outcome is not "a review":
|
||||||
|
HIGH under a low-trust preset is `fail_secure`. Report-only means the text is
|
||||||
|
never mutated — it does not mean the finding cannot block.
|
||||||
|
|
||||||
- **A connection-string password longer than 256 chars is not matched.** The
|
- **A connection-string password longer than 256 chars is not matched.** The
|
||||||
password run in the `*-connstr` egress patterns is bounded by
|
password run in the `*-connstr` egress patterns is bounded by
|
||||||
`MAX_CONNSTR_VALUE`; unbounded, it sits in front of a mandatory `@` and makes
|
`MAX_CONNSTR_VALUE`; unbounded, it sits in front of a mandatory `@` and makes
|
||||||
crafted input quadratic. Excluding the anchor character instead — the fix the
|
crafted input quadratic. Excluding the anchor character instead — the fix the
|
||||||
active-content table uses — is unavailable here because that character is `/`,
|
active-content table uses — is unavailable here because that character is `/`,
|
||||||
and a password containing `/` is the common case. The realistic long value (a
|
and a password containing `/` is the common case. **What the residual actually
|
||||||
token used as a DB password) is still caught by `egress:jwt-token`.
|
costs, measured at the 257-char boundary:** a generic long password still trips
|
||||||
|
`entropy:base64-blob` at CRITICAL, so the disposition is unchanged. A *JWT* used
|
||||||
|
as a DB password is the case that moves — the remaining detections
|
||||||
|
(`entropy:base64-blob` HIGH, `egress:jwt-token` MEDIUM) top out below CRITICAL,
|
||||||
|
so the any-tier CRITICAL block is lost: under `PRESET_TRUSTED_SOURCE` such a
|
||||||
|
document drops from `fail_secure` to `quarantine_review`. Under
|
||||||
|
`PRESET_USER_UPLOAD` it still `fail_secure`s. The credential is never silently
|
||||||
|
missed; on one preset it is held for review instead of halted.
|
||||||
|
|
||||||
## The six documented gaps (tracked by the coverage matrix)
|
## The six documented gaps (tracked by the coverage matrix)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "llm-ingestion-guard"
|
name = "llm-ingestion-guard"
|
||||||
version = "0.3.1"
|
version = "0.3.2"
|
||||||
description = "A minimal, dependency-light defensive layer for LLM ingestion pipelines — the write-time siblings of query-time chatbot guardrails."
|
description = "A minimal, dependency-light defensive layer for LLM ingestion pipelines — the write-time siblings of query-time chatbot guardrails."
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.10"
|
requires-python = ">=3.10"
|
||||||
|
|
|
||||||
|
|
@ -59,7 +59,7 @@ from .grounding import (
|
||||||
)
|
)
|
||||||
from . import okf
|
from . import okf
|
||||||
|
|
||||||
__version__ = "0.3.1"
|
__version__ = "0.3.2"
|
||||||
|
|
||||||
|
|
||||||
# --- §6 bookends: the two library-side halves around the transform ---------
|
# --- §6 bookends: the two library-side halves around the transform ---------
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue