open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

fix(divergence): our own iframe number read 3x low, and the reported cause was not the cause

Browse source 
No data file changed and no pattern moved. A published figure was wrong.

llm-ingestion-pipeline-security flagged it (coord, 2026-08-11T19:51:55Z) by
measuring the hybrid-xss:iframe-src row themselves instead of citing ours, and
got roughly 4x our number. They attributed the gap to measurement surface -
their composed scan_lexicon() against our standalone regex - and said no
re-measure was needed.

Re-measured anyway, because the claim was about OUR number. Their diagnosis does
not hold: our standalone 100 000-char figure is 7.86 s against their composed
8.95 s, so the two surfaces differ by far less than the error did. Standalone,
Python 3.14.0, same unit the document claims:

  iframe-src [^>]*    822.7 ms @ 32k     51 477.4 ms @ 256k
  published           119.6 ms @ 32k     16 857    ms @ 256k
  script-tag [^>]*     87.4 ms @ 32k      5 222.6 ms @ 256k
  published            21    ms @ 32k      5 440    ms @ 256k

So the Python script-tag figure at 256k reproduces and the one at 32k does not,
and the iframe-src pair reproduces at neither point. Error ratios are not
constant, so a single mis-sized input does not explain it, and the original
harness lived in a previous session's scratchpad and is gone. Recorded as NOT
DIAGNOSABLE rather than given a plausible cause - a guessed cause would read
like a finding.

Superseded figures are struck in place rather than overwritten. Anyone who cited
the old number needs to be able to discover that they did.

The 0.4.0 decision does not depend on this. Every corrected figure is larger, the
shape is unchanged (quadratic, x4 per doubling), and both [^><]* forms stay flat.
The 0.4.0 CHANGELOG section still quotes the old figure and is left alone: that
section is the record of what was released, not a live claim.

Verified: JSON well-formed, specs normative, charter clean, README vendoring
examples and prose moved to v0.4.1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLEZ4XCSnSrQUFA8SzkQB4
This commit is contained in:
Kjell Tore Guttormsen 2026-08-11 22:02:15 +02:00
commit 2d86151e3b
3 changed files with 67 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

32
docs/lexicon-port-divergence.md
View file

@ -295,6 +295,35 @@ scan-position quadratic, not exponential backtracking: `<script` matches at ~n p
`[^>]*` scans to end of input at each one. Nothing here is catastrophic backtracking, and
nothing here needs a nested quantifier to happen.
> **Correction, 2026-08-11 (commons `v0.4.1`) — two of the Python figures above read low, and
> the `iframe-src` figure below read low by ~3×.** The guard flagged it (coord,
> 2026-08-11T19:51:55Z) after measuring the `iframe-src` row themselves and getting roughly 4×
> our published number. They attributed the gap to measurement surface — their composed
> `scan_lexicon()` against our standalone regex. **That is not the explanation.** Re-measured
> here standalone, the same surface the numbers above claim, Python 3.14.0, median-stable across
> repeated runs (the two 256 000-char `iframe-src` runs landed 3 ms apart in 51 s):
>
> | chars | `script-tag` `[^>]*` | `script-tag` `[^><]*` | `iframe-src` `[^>]*` | `iframe-src` `[^><]*` |
> |---|---|---|---|---|
> | 32 000 | 87.4 ms | 0.17 ms | 822.7 ms | 0.25 ms |
> | 100 000 | — | — | 7 859.5 ms | — |
> | 256 000 | 5 222.6 ms | 1.25 ms | 51 477.4 ms | 2.08 ms |
>
> So: the Python `script-tag` figure at 256 000 chars **reproduces** (5.44 s published against
> 5.22 s measured), and the one at 32 000 chars does **not** (0.021 s against 0.087 s). The
> `iframe-src` pair below reproduces at neither point. The error ratios are not constant, so a
> single mis-sized input does not explain it, and the original harness lived in a previous
> session's scratchpad and is gone — **the cause is not diagnosable and is not guessed at here.**
> Our standalone 100 000-char figure (7.86 s) sits close to the guard's composed 8.95 s, which
> is why their surface hypothesis does not survive: the two surfaces differ by much less than
> the error did.
>
> **What does not change:** the shape (quadratic, ×4 per doubling), the direction, and the
> decision. Every corrected figure is *larger*, so the case against the old forms is stronger
> than the one originally published, and the new forms are flat under both engines. The guard
> flagged this because a low number could matter if either side later pins a bound — they were
> right to, and the number they should pin against is the one in this box.
**Two of the 83 patterns have this shape, not one.** Counted over the whole file, 8 patterns
carry a bare `[^>]` and none carries `[^><]`. Six of the 8 are the html-obfuscation family and
are **bounded** (`[^>]{1,256}`) — measured linear (51 → 164 ms across a 8× input growth), so
@ -304,7 +333,8 @@ other two are unbounded `[^>]*`:
- `hybrid-xss:script-tag` — the row this file already tracks.
- `hybrid-xss:iframe-src` — **not previously named by anyone**, found here by counting rather
than by being told. Same shape, same result: 119.6 ms → 16 857 ms over 32 000 → 256 000
chars of `'<iframe '`.
chars of `'<iframe '`. **Both figures are superseded — see the correction box above:
822.7 ms → 51 477.4 ms on re-measurement, standalone, same unit.**
What this does and does not license: