open/llm-security
1
0
Fork 0
Code Issues Releases 1 Activity

Squashed 'scanners/commons/' changes from 4641a7b..7ce0ba7

Browse source 
7ce0ba7 docs(carriers): the third verdict exists, and publishing an alias is what takes it away
302625e fix(conformance): the tag carrier has no output: label, and our blocker claimed it did
fe5e6b2 docs(conventions): the merge button is off for a reason, and the reason now lives in the repo
2d86151 fix(divergence): our own iframe number read 3x low, and the reported cause was not the cause
daa7ba4 release(0.4.0): two values moved by two mechanisms, and the difference is the release
2eee7e1 feat(lexicon): both unbounded rows narrow to [^><]*, and the mechanism is new here
d467324 feat(signatures): the staleness we disclosed is closed by reading the module, not the message
4187715 docs(divergence): our own form has a number now, and it is quadratic
0e765a0 docs(security): the attack surface here is data, so the report route had to say where a wrong entry gets fixed
d96fbbf docs(divergence): the span row had one witness; llm-security measured five more

git-subtree-dir: scanners/commons
git-subtree-split: 7ce0ba706cadd032ec3c16622dcfdb5ce4dc32d6
This commit is contained in:
Kjell Tore Guttormsen 2026-08-13 20:08:20 +02:00
commit bc23b07cf9
8 changed files with 844 additions and 41 deletions
Download patch file Download diff file Expand all files Collapse all files

287
CHANGELOG.md
View file

@ -9,9 +9,292 @@ Versioning note: the repository tag versions **the contract** (file set, key nam
case ids, disposition semantics). Each JSON file additionally carries its own
`"version"` field, bumped when that file changes.
## [Unreleased]
## [0.4.3] — 2026-08-11
Nothing yet.
**No fixture, id or `expected.json` moved — a runtime that passes `0.4.2` passes `0.4.3`
unchanged. What changed is a claim this repository made about a runtime it does not own, and the
claim was wrong on the day it was written.**
### Fixed
- `conformance/manifest.json` `0.3.2` → `0.3.4` — the `scope_planned.blockers` entry for
`codepoints/carriers.json` described the guard as emitting two stage-coupled labels per carrier,
"the same split for bidi and unicode-tag". The artifact-side label for tags is
`lexicon:unicode-tags-present`, emitted from `lexicon.py`, and `output.py` carries a comment
saying it deliberately does not repeat it there. **Wrong when written, not stale:** checked at
`e671edb` — the commit the sibling secret-egress blocker was measured against — where
`coverage.py` already asserted that label, and re-measured at `a59184b`.
The correction moves the blocker rather than removing it. The guard's `Finding` carries a
`detector` field beside `label`, and the prefix is that field's value: `detector="lexicon"` on
`lexicon:unicode-tags-present`, `"output"` on `output:zero-width-present`. **The prefix names
the detector, not the pipeline stage** — and for tags a single detector serves both entry
points, which is why there is no sixth `output:` label to find. "A commons id would have to be
invented stage-neutral" was never the problem. Six labels exist to adopt verbatim, the way the
83 lexicon ids were adopted from this same runtime's port.
What blocks adoption is measured and named instead, at llm-security `47905da`: `sanitize:`
asserts a strip that runtime does not perform (`scanners/unicode-scanner.mjs` exports one entry
point, `scan(targetPath, discovery)`, reporting presence with `scanner: 'UNI'`, a prose title
and no id); three of the six name a persist gate it does not have, which the corpus already has
a verdict for — §1.1 `not-applicable`, attaching to a declared **table** — but which
**publishing the alias is what takes away**: that runtime's suite walks each vendored file for
any node carrying `aliases.llm_security` and asserts every registered table is declared, so one
aliased carrier id forces `codepoints/carriers.json` into a declared set of what is today the
lexicon alone, obliging it to run all six cases and converting the three artifact-side ones into
failures; and the entry point pinned for it in `measurement.runtimes` (`scanForInjection`) does not
reach carriers at all, so carrier cases need a per-scope entry point this manifest expresses
nowhere. Both runtimes already build their carrier sets from `codepoints/carriers.json`, so the
divergence is in what a finding is *called* and where it can be *observed*, never in which code
points are carriers.
### Asked, not decided
- The three objections went to both runtimes over coord on 2026-08-11 as a decision request, each
asked the question only it can answer. **Nothing was minted.** A case id is contract surface
consumers pin against, and publishing a single carrier alias is itself irreversible — it widens
another runtime's declared table set by force of that runtime's own test suite. Minting first
would have made a proposal into a fait accompli. The manifest records the request, so a later
reader can tell "asked, unanswered" from "nobody asked".
- **A correction followed the request the same day, on our own error.** The request asserted that
the corpus had no third verdict for a case a runtime cannot reach. It has one — §1.1
`not-applicable` — and this repository wrote that section. The question was put before its own
normative spec was re-read; the follow-up says so to both runtimes and restates the choice as
mint-input-side-only, accept three standing failures, or publish a guard-only id space with no
`llm_security` alias at all.
## [0.4.2] — 2026-08-11
**No data file changed and no pattern moved.** A runtime that passes `0.4.1` passes `0.4.2`
unchanged; there is nothing here to re-measure. What the release adds is the rule set an outside
contributor could not previously read — including the reason the forge surface is shaped the way
it is.
### Added
- `CONVENTIONS.md` — the whole rule set a change here is held to, consolidated. **Not new
policy:** the charter lives in `CLAUDE.md`, the versioning and vendoring rules in `README.md`,
the reporting route in `SECURITY.md`, and the file conventions were visible only in the shape
of the files. Collected because a convention that exists only in the maintainer's head is not
one an outside reader can meet.
Two things in it were previously inferable at best. **Why pull requests are off:** this
repository is vendored into independent runtimes that pin a tag, so a change to detection data
changes what they *find*, and that has to be coordinated with each consumer **before it
exists** — which a merge button cannot do. `org-ops` reached that conclusion on 2026-08-11
from a README line, and the conclusion was right; this file is the ground it was missing.
**When a value may change:** the three mechanisms that have moved one so far — re-extraction,
retraction, and owner-directed authoring — each named with the `source_fidelity` key that
records it, and merit named explicitly as *not* on that list.
It also carries the four offline checks that stand in for the CI this organisation does not
have. Each was confirmed to go **red** on a violation, not merely green on a clean tree: a
JSON file with no `version`, a `spec/` file with no normative marker, and a planted `.sh` were
each detected. A check that cannot fail proves nothing. The checks are shell one-liners rather
than a script because a script would be `.sh`, and check 4 would fail on the tooling meant to
enforce it.
- `README.md` — a short **Contributing** section pointing at it, carrying the pull-request answer
inline so a reader who never opens the file still gets it. Same pattern the
**Reporting a wrong entry** section followed for `SECURITY.md` in `0.3.1`.
The four `v0.4.1` references in the install block and the layout table move to `v0.4.2`.
This closes the second half of what `org-ops` recorded as missing against the org standard on
2026-08-11. `SECURITY.md` was the first half, in `0.3.1`.
## [0.4.1] — 2026-08-11
**No data file changed and no pattern moved. A number this repository published was wrong, and
it was wrong in our favour's opposite direction — the corrected figures are larger.** A runtime
that passes `0.4.0` passes `0.4.1` unchanged.
### Fixed
- `docs/lexicon-port-divergence.md` (informative) — the ReDoS figures for
`hybrid-xss:iframe-src` read **~3× low**, and the Python `script-tag` figure at 32 000 chars
read ~4× low. Flagged by `llm-ingestion-pipeline-security` (coord, 2026-08-11T19:51:55Z), who
measured the row themselves rather than citing ours.
Their diagnosis was measurement surface — their composed `scan_lexicon()` against our
standalone regex. **Checked, and that is not the cause:** our standalone 100 000-char figure
(7.86 s) sits close to their composed 8.95 s, so the two surfaces differ by far less than the
error. Re-measured standalone, Python 3.14.0: `iframe-src` `[^>]*` is 822.7 ms at 32 000 chars
and 51 477.4 ms at 256 000, against the published 119.6 ms and 16 857 ms. The Python
`script-tag` figure at 256 000 chars *does* reproduce (5.44 s published, 5.22 s measured); the
one at 32 000 chars does not (0.021 s against 0.087 s).
The error ratios are not constant, so a single mis-sized input does not explain it, and the
original harness lived in a previous session's scratchpad and no longer exists. **The cause is
recorded as not diagnosable rather than guessed at.** The correction is a box in the document
carrying the re-measured table, and the superseded figures are struck in place rather than
quietly overwritten — a consumer who cited the old number needs to find out that they did.
Nothing about the `0.4.0` decision depends on this. Every corrected figure is larger, the
shape is unchanged (quadratic, ×4 per doubling), and both `[^><]*` forms remain flat under
both engines. The `0.4.0` entry below still quotes the old `iframe-src` figure; it is left as
published, because that section is the record of what was released.
## [0.4.0] — 2026-08-11
**Two detection values changed, by two different mechanisms, and the difference between those
mechanisms is the point of the release.** One pattern table was **re-extracted** from a pinned
upstream commit, the way every value in this repository has moved until now. Two lexicon rows were
**authored here at the source owner's direction**, which has never happened before and required a
reason that is not "we measured it and we were right."
A runtime that vendors this repository will see findings change. Any consumer asserting
byte-identity against `v0.3.0` goes red by construction — `lexicon/injection-lexicon.json` changed
pattern text. Ids, labels, aliases, family membership, case ids and every count are unchanged.
### Changed
- `lexicon/injection-lexicon.json` `0.7.0` → `0.8.0` — **`hybrid-xss:script-tag` and
`hybrid-xss:iframe-src` narrow their unbounded negated class from `[^>]*` to `[^><]*`.** Both
forms are quadratic in scan length on input that repeats the tag prefix and never supplies a
`>`: each occurrence is a match start and `[^>]*` scans to end of input from each one. Measured
in Node v25.8.2 at 16k / 32k / 64k / 128k / 256k chars — script-tag 32.65 / 113.36 / 479.02 /
1988.83 / **7772.25** ms, iframe-src 39.23 / 131.76 / 574.94 / 2469.55 / **9449.94** ms, ×4 per
doubling for both. Under `[^><]*` the same inputs cost 0.08–0.66 ms and 0.10–1.00 ms: flat, not
merely faster.
**Why this is not commons correcting seed data.** The dependency direction inverted. As of
`llm-security` `be14867` their four injection tables are built from this file and hold zero local
regex literals — measured on their published surface at `47905da`, with their vendored copy of
the lexicon confirmed byte-identical to `0.7.0`. So re-extraction was not available as a
mechanism: there is no upstream literal left to re-read. They re-measured the finding rather than
accepting it, rejected `[^>]{0,256}` because a bound is paddable and `[^>]{1,256}` because it
drops bare `<script>` and two corpus cases with it, chose `[^><]*`, and asked commons to carry
it. Recorded in a new `source_fidelity.owner_directed_changes` block — deliberately **not** in
`post_extraction_drift`, which would have said the source moved and commons followed, when the
source now reads commons.
Not decided by majority. The guard reached `[^><]` first and independently (`cff0437`), so all
three runtimes now agree, but a 3-of-3 count is not what moved this value and would not have
been sufficient. The justification is the same one that kept commons on `[^>]` through `0.7.0`:
this file tracks its declared source, and the declared source chose.
Accepted cost, stated plainly: content carrying a literal `<` between the tag name and the `>`
(`<script <x>`, `<script<div>`) stops matching. Measured across **all 90** conformance cases,
not only the four that cite these ids: zero lost a match, zero gained one. The dropped class is
real and unwitnessed by the corpus.
- `signatures/secret-egress.json` `0.2.0` → `0.3.0` — **the one-entry staleness disclosed in
`0.2.0` is closed by re-extraction, 18 → 19 patterns.** `OpenAI Legacy API Key`
(`\bsk-[A-Za-z0-9]{20}T3BlbkFJ[A-Za-z0-9]{20}\b`) enters at `order` 17, second to last; `JWT
(three-part token)` moves to 18 and stays last, which `ordering.last_entry_is_load_bearing`
requires.
Read out of the module text at a pinned public commit, never transcribed from the coord message
that reported it — the message contained the regex, and that is exactly the path
`evidence_limits` had ruled out. `refs/heads/main` on the public remote is `47905da`; `088e458`
carries the entry and was confirmed an ancestor with `git merge-base --is-ancestor` rather than
read off their log.
A side effect worth more than the entry: **all 19 positions were compared against the module** —
name, regex source, flags, order — with 0 divergences. Positions 0–16 came from a 2026-08-09
transcription whose module fidelity had stood recorded as llm-security's assertion rather than a
reproduced result. It is now reproduced, and that `evidence_limits` bullet is retired along with
the staleness one.
- `conformance/manifest.json` `0.3.1` → `0.3.2` — the `secret-egress` blocker prose said its note
would stand until the re-extraction landed. It landed, so item (2) (`openai-api-key-legacy` is a
real hole here) is marked closed, and the entry count moves 18 → 19. **The blocker itself does
not close**: 19 entries against the guard's 25 at different cut points is still a table
reconciliation nobody has performed, and one closed hole is not that reconciliation.
- `docs/lexicon-port-divergence.md` (informative) — the `[^>]` vs `[^><]` row gets the number
it never had for **this** side, and then gets closed. The guard disclosed that their `[^><]` is a
measured ReDoS fix (`cff0437`), not a preference, and asked commons to measure its own form
rather than take their word. Measured here in Node and Python: `<script\b[^>]*>` is **quadratic**
in scan position on input that denies it a `>` — ×4.0 per doubling, 6.7 s at 256 000 chars
against 0.41 ms for the guard's form.
Counted across the whole lexicon rather than assumed from the one row: 8 of 83 patterns carry
a bare `[^>]`, six of them bounded (`{1,256}`, measured linear) and **two unbounded**. The
second unbounded one, `hybrid-xss:iframe-src`, had not been named by any party — 16.9 s at
256 000 chars.
The finding was routed to the owning runtime, which is what `SECURITY.md` promises an outsider
would happen — the first time that route was walked from the inside. It came back as a decision
the same day, and the document now records the closure with the invariant intact: the
measurement travelled, the authority did not. Had `llm-security` declined, both rows would still
read `[^>]*` and this file would record a permanent divergence instead.
## [0.3.1] — 2026-08-11
**No pattern changed value. One shipped table is disclosed as stale, and the repository gains
the reporting route it did not have.** Nothing in `patterns`, `expected.json` or any id moved,
so a runtime that passes `0.3.0` passes `0.3.1` unchanged. Read the first entry anyway if you
vendor `signatures/secret-egress.json`: it now says, in the file, that it under-matches its own
source by one entry.
### Added
- `SECURITY.md` — the reporting route for a repository whose attack surface is **data**. It
answers the question an ordinary security policy does not have to: how to report that a
*detection-table entry is wrong*, and why a confirmed defect in extracted data is decided in
the runtime it was extracted from before it is changed here. Names what is in scope (a silent
false negative, a fixture that sanctions a miss, an unsafe normative clause, a secret in the
history, data gone stale against its source), what is a documented boundary rather than a
vulnerability, and the two classes that skip the routing — a real secret, and data authored
here rather than extracted. States plainly that fix latency is bounded by the owning runtime's
schedule and the consumer's pull, not by this repository's.
Written because `org-ops` recorded the file as missing against the org standard on
2026-08-11, and because four files here are detection data where a mistake is a detector that
looks like it works. `CONVENTIONS.md`, recorded in the same message, is not in this release.
- `README.md` — a short **Reporting a wrong entry** section pointing at it. Without it the
policy is a file nobody looking at the front page would know to open.
### Changed
- `signatures/secret-egress.json` `0.1.0` → `0.2.0` — **a staleness disclosure, not a data
change.** All 18 patterns are byte-identical to `0.1.0`; one entry is added to
`provenance.evidence_limits`. `llm-security` reports having taken the source `SECRET_PATTERNS`
from 18 to 19 by adding `OpenAI Legacy API Key`. That is recorded as their report and
explicitly **not** reproduced here — the commit carrying it is not on their public remote,
which was measured at `b1ba1fb` on 2026-08-11. What *was* measured here: none of the 18
patterns matches a legacy `sk-…T3BlbkFJ…` key shape. So a consumer vendoring this file
under-matches the seed hook by one entry, on a live credential shape, and now reads that in the
file rather than inferring it. It will be closed by re-extraction from a pinned public commit,
never by authoring the entry here from a message.
- `conformance/manifest.json` `0.3.0` → `0.3.1` — the `scope_planned.blockers` text for
`signatures/secret-egress.json` is corrected. Through `0.3.0` it ended by naming
`gcp-service-account-json` and `openai-api-key-legacy` together as ids "absent here". They are
two different kinds of fact, and one of them was misleading.
Measured 2026-08-11, by running this file's own 18 patterns in `order` over a service-account
document, against the guard at commit `e671edb`: a **complete** GCP service-account key file
*is* matched here, at order 11 (`Private Key PEM Block` — its `(?:RSA |EC |DSA |OPENSSH )?`
prefix group is optional, so the bare PKCS#8 header such a file carries matches). The same
document with `private_key` removed matches nothing here while the guard's marker pattern still
fires. That is a **cut-point** difference — the guard detects the document marker, this table
detects the key material — which is what the blocker is about, and not a missing entry.
`openai-api-key-legacy`, by contrast, is a real hole here today, and is now recorded as one.
The correction is folded into the existing blocker string rather than added as a sibling key:
`blockers` is a map from table path to text, and a second key under a table path would read as
a second table to anything iterating it.
- `docs/lexicon-port-divergence.md` (informative) — the residual `[^>]` vs `[^><]` row gains a
fuller witness set. `llm-security` measured the three forms as **totally ordered** by what they
match, each a strict superset of the next, and named two input classes the guard's narrower
class drops (`<script a="<" >x`, `<script<div>`) beyond the one commons had recorded.
Reproduced here independently, including the strict-superset property in both directions,
before being written down.
Their argument — that the narrower class buys an empty false-positive surface, since anything
reaching `[^>]`-and-not-`[^><]` already contains a literal `<script` tag — is recorded as
theirs and explicitly **not** what decided commons' form. Commons carries `[^>]` on provenance,
and would have carried `[^><]` had the source been the guard's. Also records that they asked to
hear the guard's reason for `[^><]` before commons shipped and commons shipped first, with why
that order was deliberate.
No data file touched; `v0.3.0` is unaffected.
## [0.3.0] — 2026-08-11