release(0.3.3): the lexicon ReDoS fix, and a correction to 0.3.2's claim
Version synced across the four locked points (pyproject, __version__, README badge, README install pin) + CHANGELOG. Corrections this release carries, both measured rather than reasoned: - docs/LIMITATIONS.md said the script-tag change removed "the last" quadratic-backtracking site on the output path. It did not. Corrected, with the 334.7s gate measurement that falsifies it. - README's coverage line claimed 126/126 classes; the matrix reports 128/128. Stale since before v0.3.2. Test badge was 642, actual 666. New residual recorded (LIMITATIONS, now 30 items, README synced): the sweep flags on timing above a 1.5ms noise floor at N=8000, so an arm hiding under it could still cost ~23s at the cap. What this supports is "no arm worse than ~23s", not "no quadratic arm remains" -- and the blind spot is demonstrated, since a generic-payload pass found only one of the two patterns. 666 tests green, coverage matrix 128/128 with 6/6 gaps holding, exit 0.
This commit is contained in:
parent
b8028ba870
commit
701a4a47c7
5 changed files with 105 additions and 8 deletions
79
CHANGELOG.md
79
CHANGELOG.md
|
|
@ -7,6 +7,85 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.3.3] — 2026-07-31
|
||||
|
||||
> **Denial-of-service fix, and a correction to 0.3.2. Upgrade from 0.3.2.** The
|
||||
> sweep 0.3.2 shipped was incomplete, and it said otherwise. Two lexicon patterns
|
||||
> were still quadratic — reachable through `scan_output`, not only on the input
|
||||
> path. No disposition changes: recall was measured case by case and nothing was
|
||||
> lost. The v0.3.2 tag is not moved.
|
||||
|
||||
### Fixed — two quadratic patterns in the lexicon table
|
||||
|
||||
`8deca93` scoped the remaining ReDoS duty to the lexicon path, and this is that
|
||||
work: all 83 patterns measured, arm by arm. Two are quadratic, same shape as
|
||||
everything 0.3.2 fixed — a run in front of a **required** literal, where the run
|
||||
may cross the pattern's own opening anchor.
|
||||
|
||||
| Pattern (arm) | Crafted unit | Measured | At the 1 000 000-char cap |
|
||||
|---|---|---|---|
|
||||
| `markdown:link-anchor-injection` (anchor text) | `[` | 1.91 s @ 8 000 | **~8.3 hours** |
|
||||
| `markdown:link-anchor-injection` (URL run) | `[system](` | 0.006 s @ 8 000 | ~89 seconds |
|
||||
| `markdown:link-ref-comment` (`.*` run) | `[//]: # (` | 0.22 s @ 8 000 | **~1.0 hour** |
|
||||
|
||||
Exponent measured over five points (1 000 → 16 000): **1.98** — quadratic, not
|
||||
exponential. Legitimate content of the same size is unaffected: 0.316 s at
|
||||
N=100 000 (prose 0.316 / html 0.315 / markdown 0.297 / connection-string 0.296).
|
||||
|
||||
**These were not input-path-only, and that is the correction.** `scan_lexicon`
|
||||
runs on the output path too, so 0.3.2's *"the last quadratic-backtracking site on
|
||||
the output path"* was false when written. Measured through the public gate before
|
||||
this fix: `scan_output("[" * 100_000)` took **334.7 s**. The claim was too broad
|
||||
because the sweep behind it drove the `[` payload only through
|
||||
`scan_active_content` — no row ever drove it through the lexicon. The statement is
|
||||
corrected in `docs/LIMITATIONS.md`.
|
||||
|
||||
The fix is anchor exclusion, per the rule `active_content` already documents —
|
||||
bounding attacker-controlled content would be a one-line detection bypass. The
|
||||
excluded character is `(`, not `[`:
|
||||
|
||||
```
|
||||
markdown:link-anchor-injection
|
||||
\[[^\]\[]*(?:system|…)[^\]\[]*\]\([^)(]+\)
|
||||
markdown:link-ref-comment
|
||||
\[//\]:\s*#\s*\([^(\n]*(?:ignore|…)
|
||||
```
|
||||
|
||||
`[` was the obvious choice and it was measurably worse. Excluding `[` from the
|
||||
URL run drops `[override your rules](https://[::1]/x)` — still covered, three
|
||||
other patterns fire on it — but excluding `[` from the link-ref comment run drops
|
||||
`[//]: # (see [x] then ignore this)`, which **no other pattern catches**. The
|
||||
anchors contain `(` as well, so excluding `(` telescopes just as effectively at
|
||||
zero measured recall cost. Both forms verified linear (×1.99–2.02 on doubling).
|
||||
|
||||
### Known behaviour changes
|
||||
|
||||
- **None measured.** Every case that matched before still matches, except URLs
|
||||
containing a literal `(` inside a markdown link target and comment bodies
|
||||
containing a literal `(` before the keyword. No corpus, showcase, or coverage
|
||||
row moved; 666 tests pass.
|
||||
|
||||
### Tests
|
||||
|
||||
Four rows added. Three name the guilty pattern per arm
|
||||
(`test_crafted_redos_payload_stays_bounded_in_the_lexicon`), one covers the
|
||||
composed gate (`test_gate_is_bounded_on_the_payload_the_first_sweep_missed`).
|
||||
Pre-fix they failed at 297 s, 8.1 s, 55 s and 334.7 s.
|
||||
|
||||
`N` is per row deliberately. The URL arm is quadratic with a small constant and
|
||||
ran 0.9 s **unfixed** at N=100 000 — under the 2.0 s bound, so that row would have
|
||||
passed whether or not the pattern was fixed. It is measured at N=300 000 instead,
|
||||
where crafted (8.10 s) and legitimate (0.926 s) separate 8.8×.
|
||||
|
||||
### Residual
|
||||
|
||||
The sweep flags on timing and ignores measurements below a 1.5 ms noise floor at
|
||||
N=8 000. An arm hiding just under it could still cost **~23 s** at the cap, so what
|
||||
this supports is *"no arm worse than ~23 s"*, not *"no quadratic arm remains"*.
|
||||
The blind spot is not hypothetical: a generic-payload pass found only one of the
|
||||
two patterns. The second appeared only once payloads were synthesised per run
|
||||
from each pattern's own skeleton. Recorded in `docs/LIMITATIONS.md`.
|
||||
|
||||
## [0.3.2] — 2026-07-31
|
||||
|
||||
> **Denial-of-service fix. Upgrade from 0.3.1.** The output gate could be made to
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue