open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

feat(lexicon): both unbounded rows narrow to [^><]*, and the mechanism is new here

Browse source 
injection-lexicon.json 0.7.0 -> 0.8.0. hybrid-xss:script-tag and
hybrid-xss:iframe-src drop the `>`-only negated class for `[^><]*`.

Both were quadratic in scan length on input repeating the tag prefix without
ever supplying a `>`: every occurrence is a match start and `[^>]*` runs to end
of input from each one. Node v25.8.2, 16k/32k/64k/128k/256k chars - script-tag
32.65/113.36/479.02/1988.83/7772.25 ms, iframe-src
39.23/131.76/574.94/2469.55/9449.94 ms, x4 per doubling. Under `[^><]*` the same
inputs cost 0.08-0.66 ms and 0.10-1.00 ms. Flat, not merely faster.

This is NOT re-extraction and it is not commons correcting seed data. The
dependency direction inverted: at llm-security be14867 their four injection
tables are built from this file and hold zero local regex literals - measured on
their published surface at 47905da, their vendored lexicon confirmed
byte-identical to 0.7.0. There is no upstream literal left to re-read, so
re-extraction was unavailable as a mechanism. They re-measured rather than
accepting our report, rejected [^>]{0,256} because a bound is paddable and
[^>]{1,256} because it drops bare `<script>` along with two corpus cases, chose
[^><]*, and asked us to carry it.

Recorded in a new source_fidelity.owner_directed_changes block, deliberately not
in post_extraction_drift - that block's shape asserts the source moved and
commons followed, which would be false in the one way that matters now that the
source reads commons.

Not majority. The guard reached [^><] first and independently (cff0437), so all
three agree, but 3-of-3 is not what moved this value. The justification is the
one that kept us on [^>] through 0.7.0 and is quoted in the divergence doc: this
file tracks its declared source, and the declared source chose.

Cost, stated: `<script <x>` and `<script<div>` stop matching. Measured over all
90 conformance cases rather than the 4 citing these ids - 0 lost, 0 gained. The
dropped class is real and unwitnessed by the corpus.

count_against_llm_security_head is now undefined rather than 83: its comparison
target no longer exists. A diff against their vendored copy is a subtree
freshness check, not an independent agreement measurement, and the field says so.

Verified against the SHIPPED file, patterns read from the JSON and never from
literals in the harness: 83/83 compile in Node bare and `u`; whole-table ReDoS
budget at the guard's 512 KB cap, per-pattern unit derived from each pattern's
own literal prefix, slowest of 83 is 1.90 ms against a 250 ms budget; all 4
corpus cases citing the changed ids still match. Gate proven discriminating, not
assumed: the pre-0.8.0 forms measure 37 374 ms and 44 514 ms on that same check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLEZ4XCSnSrQUFA8SzkQB4
This commit is contained in:
Kjell Tore Guttormsen 2026-08-11 21:52:43 +02:00
commit 2eee7e19ee
2 changed files with 66 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

45
docs/lexicon-port-divergence.md
View file

@ -5,6 +5,11 @@ records a measured disagreement between two ports of one source table, so the de
taken where each table is tested. Under this repository's behaviour-preservation invariant,
a divergence found here is **reported, not fixed**.
That invariant held through the closure recorded below. Two rows this document measured did change
in `lexicon/injection-lexicon.json` `0.8.0` — and not because this document found them wrong. The
finding was routed to the runtime that owns the value, that runtime re-measured it, decided, and
asked commons to carry the result. The measurement travelled; the authority did not.
Produced 2026-08-09. Every number below came from a command; the scripts live in the session
scratchpad rather than in this repository, because executable code here would breach the
charter. They are reproducible from the method column.
@ -166,11 +171,16 @@ The direction matters for what it cannot break: the new form matches a strict su
one, so relative to `0.6.0` it can add matches and cannot add a false negative. The reverse change
would not have been adoptable on the same reasoning.
**What is still open is the one-byte span difference**, and convergence did not touch it. The
guard excludes `<` from its negated class and the other two do not. The three forms turn out to
be **totally ordered by what they match** — each a strict superset of the next:
**The one-byte span difference is CLOSED as of commons `0.8.0`** — all three runtimes now carry
`[^><]*`. What follows is the account of the divergence while it was open, kept because the table
below is the measurement that made the closure decidable, and because a consumer diffing against
`v0.3.0` will find the two changed rows and needs the coordinate.
| Witness | commons `0.7.0` / `llm-security` `[^>]` | guard `[^><]` | commons ≤ `0.6.0` |
While it was open, the guard excluded `<` from its negated class and the other two did not. The
three forms turn out to be **totally ordered by what they match** — each a strict superset of the
next:
| Witness | commons `0.7.0` / `llm-security` `[^>]` | guard `[^><]`, and commons from `0.8.0` | commons ≤ `0.6.0` |
| --- | --- | --- | --- |
| `<script>steal()</script>` | match | match | match |
| `<script src=x.js>` | match | match | no match |
@ -198,6 +208,23 @@ commons shipped first. That order is deliberate rather than an oversight: a reas
argument is a reason for the *guard's own port*, and commons would still track its declared
source. Put to the guard as an open question 2026-08-11; unanswered at time of writing.
**How it resolved, and why the resolution used the same rule rather than overturning it.** Both
halves of that open question came back the same day. The guard answered that `[^><]` is a
*measured* ReDoS fix on their side (their commit `cff0437`), not an aesthetic choice — so the
"reason that beats this argument" the paragraph above asked for turned out to exist. And commons
then measured its own copy and found the same defect in it: `<script\b[^>]*>` and
`<iframe\b[^>]*src…` are both quadratic in scan length, the second one having been named by nobody
until it was counted. `llm-security` re-measured rather than taking the report, reached `[^><]*`
independently, and asked commons to carry it — because their table is now *built from this file*,
so there is no local literal left for them to change without forking.
The paragraph above said commons "would still track its declared source." That is exactly what
`0.8.0` does: the declared source chose `[^><]*` and commons followed. The rule did not bend to
accommodate a better argument — the argument won *at the source*, which is the only place this
repository lets an argument decide a `pattern` value. The recall cost stated in the two bold rows
of the table is now paid deliberately: measured across all 90 conformance cases, zero lost a match
and zero gained one, so the dropped classes remain real and remain unwitnessed by the corpus.
## Why they diverge: two different ReDoS mitigations of one table
This is not drift, and framing it as a bug in either repository would be wrong.
@ -287,6 +314,16 @@ What this does and does not license:
had the source been the guard's — that was true before this measurement and stays true after.
- It does mean the finding is **owed to the owning runtime**, which is `llm-security` for both
patterns. Sent 2026-08-11, with these numbers and the `iframe-src` row they did not have.
**Outcome, same day:** the owning runtime reproduced both rows on its own vendored copy,
measured the exposure at its real entry-point caps (which the numbers above deliberately do not
cover — a pattern cost is not an exposure), judged it unacceptable, chose `[^><]*` over
`[^>]{0,256}` on the grounds that a bound is paddable, and asked commons to carry the change.
Commons `0.8.0` carries it. Note what did **and did not** license that: the measurement above
still licenses nothing on its own, and the two bullets stand as written. What licensed the
change is the *owner's decision*, arriving through the one channel this repository accepts for
moving a `pattern` value. Had they declined, the rows would still read `[^>]*` today and this
document would record a permanent divergence instead.
- The guard's own honest disclosure belongs on the record too: their suite passes under both
forms (717 green), because the DoS row that ought to discriminate uses `'<script>'` as its
repeating unit, which supplies the `>` and never exercises the run. The invariant is