7ce0ba7 docs(carriers): the third verdict exists, and publishing an alias is what takes it away 302625e fix(conformance): the tag carrier has no output: label, and our blocker claimed it did fe5e6b2 docs(conventions): the merge button is off for a reason, and the reason now lives in the repo 2d86151 fix(divergence): our own iframe number read 3x low, and the reported cause was not the cause daa7ba4 release(0.4.0): two values moved by two mechanisms, and the difference is the release 2eee7e1 feat(lexicon): both unbounded rows narrow to [^><]*, and the mechanism is new here d467324 feat(signatures): the staleness we disclosed is closed by reading the module, not the message 4187715 docs(divergence): our own form has a number now, and it is quadratic 0e765a0 docs(security): the attack surface here is data, so the report route had to say where a wrong entry gets fixed d96fbbf docs(divergence): the span row had one witness; llm-security measured five more git-subtree-dir: scanners/commons git-subtree-split: 7ce0ba706cadd032ec3c16622dcfdb5ce4dc32d6
864 lines
56 KiB
Markdown
864 lines
56 KiB
Markdown
# Changelog
|
||
|
||
All notable changes to this project will be documented in this file.
|
||
|
||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this
|
||
project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||
|
||
Versioning note: the repository tag versions **the contract** (file set, key names,
|
||
case ids, disposition semantics). Each JSON file additionally carries its own
|
||
`"version"` field, bumped when that file changes.
|
||
|
||
## [0.4.3] — 2026-08-11
|
||
|
||
**No fixture, id or `expected.json` moved — a runtime that passes `0.4.2` passes `0.4.3`
|
||
unchanged. What changed is a claim this repository made about a runtime it does not own, and the
|
||
claim was wrong on the day it was written.**
|
||
|
||
### Fixed
|
||
|
||
- `conformance/manifest.json` `0.3.2` → `0.3.4` — the `scope_planned.blockers` entry for
|
||
`codepoints/carriers.json` described the guard as emitting two stage-coupled labels per carrier,
|
||
"the same split for bidi and unicode-tag". The artifact-side label for tags is
|
||
`lexicon:unicode-tags-present`, emitted from `lexicon.py`, and `output.py` carries a comment
|
||
saying it deliberately does not repeat it there. **Wrong when written, not stale:** checked at
|
||
`e671edb` — the commit the sibling secret-egress blocker was measured against — where
|
||
`coverage.py` already asserted that label, and re-measured at `a59184b`.
|
||
|
||
The correction moves the blocker rather than removing it. The guard's `Finding` carries a
|
||
`detector` field beside `label`, and the prefix is that field's value: `detector="lexicon"` on
|
||
`lexicon:unicode-tags-present`, `"output"` on `output:zero-width-present`. **The prefix names
|
||
the detector, not the pipeline stage** — and for tags a single detector serves both entry
|
||
points, which is why there is no sixth `output:` label to find. "A commons id would have to be
|
||
invented stage-neutral" was never the problem. Six labels exist to adopt verbatim, the way the
|
||
83 lexicon ids were adopted from this same runtime's port.
|
||
|
||
What blocks adoption is measured and named instead, at llm-security `47905da`: `sanitize:`
|
||
asserts a strip that runtime does not perform (`scanners/unicode-scanner.mjs` exports one entry
|
||
point, `scan(targetPath, discovery)`, reporting presence with `scanner: 'UNI'`, a prose title
|
||
and no id); three of the six name a persist gate it does not have, which the corpus already has
|
||
a verdict for — §1.1 `not-applicable`, attaching to a declared **table** — but which
|
||
**publishing the alias is what takes away**: that runtime's suite walks each vendored file for
|
||
any node carrying `aliases.llm_security` and asserts every registered table is declared, so one
|
||
aliased carrier id forces `codepoints/carriers.json` into a declared set of what is today the
|
||
lexicon alone, obliging it to run all six cases and converting the three artifact-side ones into
|
||
failures; and the entry point pinned for it in `measurement.runtimes` (`scanForInjection`) does not
|
||
reach carriers at all, so carrier cases need a per-scope entry point this manifest expresses
|
||
nowhere. Both runtimes already build their carrier sets from `codepoints/carriers.json`, so the
|
||
divergence is in what a finding is *called* and where it can be *observed*, never in which code
|
||
points are carriers.
|
||
|
||
### Asked, not decided
|
||
|
||
- The three objections went to both runtimes over coord on 2026-08-11 as a decision request, each
|
||
asked the question only it can answer. **Nothing was minted.** A case id is contract surface
|
||
consumers pin against, and publishing a single carrier alias is itself irreversible — it widens
|
||
another runtime's declared table set by force of that runtime's own test suite. Minting first
|
||
would have made a proposal into a fait accompli. The manifest records the request, so a later
|
||
reader can tell "asked, unanswered" from "nobody asked".
|
||
|
||
- **A correction followed the request the same day, on our own error.** The request asserted that
|
||
the corpus had no third verdict for a case a runtime cannot reach. It has one — §1.1
|
||
`not-applicable` — and this repository wrote that section. The question was put before its own
|
||
normative spec was re-read; the follow-up says so to both runtimes and restates the choice as
|
||
mint-input-side-only, accept three standing failures, or publish a guard-only id space with no
|
||
`llm_security` alias at all.
|
||
|
||
## [0.4.2] — 2026-08-11
|
||
|
||
**No data file changed and no pattern moved.** A runtime that passes `0.4.1` passes `0.4.2`
|
||
unchanged; there is nothing here to re-measure. What the release adds is the rule set an outside
|
||
contributor could not previously read — including the reason the forge surface is shaped the way
|
||
it is.
|
||
|
||
### Added
|
||
|
||
- `CONVENTIONS.md` — the whole rule set a change here is held to, consolidated. **Not new
|
||
policy:** the charter lives in `CLAUDE.md`, the versioning and vendoring rules in `README.md`,
|
||
the reporting route in `SECURITY.md`, and the file conventions were visible only in the shape
|
||
of the files. Collected because a convention that exists only in the maintainer's head is not
|
||
one an outside reader can meet.
|
||
|
||
Two things in it were previously inferable at best. **Why pull requests are off:** this
|
||
repository is vendored into independent runtimes that pin a tag, so a change to detection data
|
||
changes what they *find*, and that has to be coordinated with each consumer **before it
|
||
exists** — which a merge button cannot do. `org-ops` reached that conclusion on 2026-08-11
|
||
from a README line, and the conclusion was right; this file is the ground it was missing.
|
||
**When a value may change:** the three mechanisms that have moved one so far — re-extraction,
|
||
retraction, and owner-directed authoring — each named with the `source_fidelity` key that
|
||
records it, and merit named explicitly as *not* on that list.
|
||
|
||
It also carries the four offline checks that stand in for the CI this organisation does not
|
||
have. Each was confirmed to go **red** on a violation, not merely green on a clean tree: a
|
||
JSON file with no `version`, a `spec/` file with no normative marker, and a planted `.sh` were
|
||
each detected. A check that cannot fail proves nothing. The checks are shell one-liners rather
|
||
than a script because a script would be `.sh`, and check 4 would fail on the tooling meant to
|
||
enforce it.
|
||
|
||
- `README.md` — a short **Contributing** section pointing at it, carrying the pull-request answer
|
||
inline so a reader who never opens the file still gets it. Same pattern the
|
||
**Reporting a wrong entry** section followed for `SECURITY.md` in `0.3.1`.
|
||
|
||
The four `v0.4.1` references in the install block and the layout table move to `v0.4.2`.
|
||
|
||
This closes the second half of what `org-ops` recorded as missing against the org standard on
|
||
2026-08-11. `SECURITY.md` was the first half, in `0.3.1`.
|
||
|
||
## [0.4.1] — 2026-08-11
|
||
|
||
**No data file changed and no pattern moved. A number this repository published was wrong, and
|
||
it was wrong in our favour's opposite direction — the corrected figures are larger.** A runtime
|
||
that passes `0.4.0` passes `0.4.1` unchanged.
|
||
|
||
### Fixed
|
||
|
||
- `docs/lexicon-port-divergence.md` (informative) — the ReDoS figures for
|
||
`hybrid-xss:iframe-src` read **~3× low**, and the Python `script-tag` figure at 32 000 chars
|
||
read ~4× low. Flagged by `llm-ingestion-pipeline-security` (coord, 2026-08-11T19:51:55Z), who
|
||
measured the row themselves rather than citing ours.
|
||
|
||
Their diagnosis was measurement surface — their composed `scan_lexicon()` against our
|
||
standalone regex. **Checked, and that is not the cause:** our standalone 100 000-char figure
|
||
(7.86 s) sits close to their composed 8.95 s, so the two surfaces differ by far less than the
|
||
error. Re-measured standalone, Python 3.14.0: `iframe-src` `[^>]*` is 822.7 ms at 32 000 chars
|
||
and 51 477.4 ms at 256 000, against the published 119.6 ms and 16 857 ms. The Python
|
||
`script-tag` figure at 256 000 chars *does* reproduce (5.44 s published, 5.22 s measured); the
|
||
one at 32 000 chars does not (0.021 s against 0.087 s).
|
||
|
||
The error ratios are not constant, so a single mis-sized input does not explain it, and the
|
||
original harness lived in a previous session's scratchpad and no longer exists. **The cause is
|
||
recorded as not diagnosable rather than guessed at.** The correction is a box in the document
|
||
carrying the re-measured table, and the superseded figures are struck in place rather than
|
||
quietly overwritten — a consumer who cited the old number needs to find out that they did.
|
||
|
||
Nothing about the `0.4.0` decision depends on this. Every corrected figure is larger, the
|
||
shape is unchanged (quadratic, ×4 per doubling), and both `[^><]*` forms remain flat under
|
||
both engines. The `0.4.0` entry below still quotes the old `iframe-src` figure; it is left as
|
||
published, because that section is the record of what was released.
|
||
|
||
## [0.4.0] — 2026-08-11
|
||
|
||
**Two detection values changed, by two different mechanisms, and the difference between those
|
||
mechanisms is the point of the release.** One pattern table was **re-extracted** from a pinned
|
||
upstream commit, the way every value in this repository has moved until now. Two lexicon rows were
|
||
**authored here at the source owner's direction**, which has never happened before and required a
|
||
reason that is not "we measured it and we were right."
|
||
|
||
A runtime that vendors this repository will see findings change. Any consumer asserting
|
||
byte-identity against `v0.3.0` goes red by construction — `lexicon/injection-lexicon.json` changed
|
||
pattern text. Ids, labels, aliases, family membership, case ids and every count are unchanged.
|
||
|
||
### Changed
|
||
|
||
- `lexicon/injection-lexicon.json` `0.7.0` → `0.8.0` — **`hybrid-xss:script-tag` and
|
||
`hybrid-xss:iframe-src` narrow their unbounded negated class from `[^>]*` to `[^><]*`.** Both
|
||
forms are quadratic in scan length on input that repeats the tag prefix and never supplies a
|
||
`>`: each occurrence is a match start and `[^>]*` scans to end of input from each one. Measured
|
||
in Node v25.8.2 at 16k / 32k / 64k / 128k / 256k chars — script-tag 32.65 / 113.36 / 479.02 /
|
||
1988.83 / **7772.25** ms, iframe-src 39.23 / 131.76 / 574.94 / 2469.55 / **9449.94** ms, ×4 per
|
||
doubling for both. Under `[^><]*` the same inputs cost 0.08–0.66 ms and 0.10–1.00 ms: flat, not
|
||
merely faster.
|
||
|
||
**Why this is not commons correcting seed data.** The dependency direction inverted. As of
|
||
`llm-security` `be14867` their four injection tables are built from this file and hold zero local
|
||
regex literals — measured on their published surface at `47905da`, with their vendored copy of
|
||
the lexicon confirmed byte-identical to `0.7.0`. So re-extraction was not available as a
|
||
mechanism: there is no upstream literal left to re-read. They re-measured the finding rather than
|
||
accepting it, rejected `[^>]{0,256}` because a bound is paddable and `[^>]{1,256}` because it
|
||
drops bare `<script>` and two corpus cases with it, chose `[^><]*`, and asked commons to carry
|
||
it. Recorded in a new `source_fidelity.owner_directed_changes` block — deliberately **not** in
|
||
`post_extraction_drift`, which would have said the source moved and commons followed, when the
|
||
source now reads commons.
|
||
|
||
Not decided by majority. The guard reached `[^><]` first and independently (`cff0437`), so all
|
||
three runtimes now agree, but a 3-of-3 count is not what moved this value and would not have
|
||
been sufficient. The justification is the same one that kept commons on `[^>]` through `0.7.0`:
|
||
this file tracks its declared source, and the declared source chose.
|
||
|
||
Accepted cost, stated plainly: content carrying a literal `<` between the tag name and the `>`
|
||
(`<script <x>`, `<script<div>`) stops matching. Measured across **all 90** conformance cases,
|
||
not only the four that cite these ids: zero lost a match, zero gained one. The dropped class is
|
||
real and unwitnessed by the corpus.
|
||
|
||
- `signatures/secret-egress.json` `0.2.0` → `0.3.0` — **the one-entry staleness disclosed in
|
||
`0.2.0` is closed by re-extraction, 18 → 19 patterns.** `OpenAI Legacy API Key`
|
||
(`\bsk-[A-Za-z0-9]{20}T3BlbkFJ[A-Za-z0-9]{20}\b`) enters at `order` 17, second to last; `JWT
|
||
(three-part token)` moves to 18 and stays last, which `ordering.last_entry_is_load_bearing`
|
||
requires.
|
||
|
||
Read out of the module text at a pinned public commit, never transcribed from the coord message
|
||
that reported it — the message contained the regex, and that is exactly the path
|
||
`evidence_limits` had ruled out. `refs/heads/main` on the public remote is `47905da`; `088e458`
|
||
carries the entry and was confirmed an ancestor with `git merge-base --is-ancestor` rather than
|
||
read off their log.
|
||
|
||
A side effect worth more than the entry: **all 19 positions were compared against the module** —
|
||
name, regex source, flags, order — with 0 divergences. Positions 0–16 came from a 2026-08-09
|
||
transcription whose module fidelity had stood recorded as llm-security's assertion rather than a
|
||
reproduced result. It is now reproduced, and that `evidence_limits` bullet is retired along with
|
||
the staleness one.
|
||
|
||
- `conformance/manifest.json` `0.3.1` → `0.3.2` — the `secret-egress` blocker prose said its note
|
||
would stand until the re-extraction landed. It landed, so item (2) (`openai-api-key-legacy` is a
|
||
real hole here) is marked closed, and the entry count moves 18 → 19. **The blocker itself does
|
||
not close**: 19 entries against the guard's 25 at different cut points is still a table
|
||
reconciliation nobody has performed, and one closed hole is not that reconciliation.
|
||
|
||
- `docs/lexicon-port-divergence.md` (informative) — the `[^>]` vs `[^><]` row gets the number
|
||
it never had for **this** side, and then gets closed. The guard disclosed that their `[^><]` is a
|
||
measured ReDoS fix (`cff0437`), not a preference, and asked commons to measure its own form
|
||
rather than take their word. Measured here in Node and Python: `<script\b[^>]*>` is **quadratic**
|
||
in scan position on input that denies it a `>` — ×4.0 per doubling, 6.7 s at 256 000 chars
|
||
against 0.41 ms for the guard's form.
|
||
|
||
Counted across the whole lexicon rather than assumed from the one row: 8 of 83 patterns carry
|
||
a bare `[^>]`, six of them bounded (`{1,256}`, measured linear) and **two unbounded**. The
|
||
second unbounded one, `hybrid-xss:iframe-src`, had not been named by any party — 16.9 s at
|
||
256 000 chars.
|
||
|
||
The finding was routed to the owning runtime, which is what `SECURITY.md` promises an outsider
|
||
would happen — the first time that route was walked from the inside. It came back as a decision
|
||
the same day, and the document now records the closure with the invariant intact: the
|
||
measurement travelled, the authority did not. Had `llm-security` declined, both rows would still
|
||
read `[^>]*` and this file would record a permanent divergence instead.
|
||
|
||
## [0.3.1] — 2026-08-11
|
||
|
||
**No pattern changed value. One shipped table is disclosed as stale, and the repository gains
|
||
the reporting route it did not have.** Nothing in `patterns`, `expected.json` or any id moved,
|
||
so a runtime that passes `0.3.0` passes `0.3.1` unchanged. Read the first entry anyway if you
|
||
vendor `signatures/secret-egress.json`: it now says, in the file, that it under-matches its own
|
||
source by one entry.
|
||
|
||
### Added
|
||
|
||
- `SECURITY.md` — the reporting route for a repository whose attack surface is **data**. It
|
||
answers the question an ordinary security policy does not have to: how to report that a
|
||
*detection-table entry is wrong*, and why a confirmed defect in extracted data is decided in
|
||
the runtime it was extracted from before it is changed here. Names what is in scope (a silent
|
||
false negative, a fixture that sanctions a miss, an unsafe normative clause, a secret in the
|
||
history, data gone stale against its source), what is a documented boundary rather than a
|
||
vulnerability, and the two classes that skip the routing — a real secret, and data authored
|
||
here rather than extracted. States plainly that fix latency is bounded by the owning runtime's
|
||
schedule and the consumer's pull, not by this repository's.
|
||
|
||
Written because `org-ops` recorded the file as missing against the org standard on
|
||
2026-08-11, and because four files here are detection data where a mistake is a detector that
|
||
looks like it works. `CONVENTIONS.md`, recorded in the same message, is not in this release.
|
||
|
||
- `README.md` — a short **Reporting a wrong entry** section pointing at it. Without it the
|
||
policy is a file nobody looking at the front page would know to open.
|
||
|
||
### Changed
|
||
|
||
- `signatures/secret-egress.json` `0.1.0` → `0.2.0` — **a staleness disclosure, not a data
|
||
change.** All 18 patterns are byte-identical to `0.1.0`; one entry is added to
|
||
`provenance.evidence_limits`. `llm-security` reports having taken the source `SECRET_PATTERNS`
|
||
from 18 to 19 by adding `OpenAI Legacy API Key`. That is recorded as their report and
|
||
explicitly **not** reproduced here — the commit carrying it is not on their public remote,
|
||
which was measured at `b1ba1fb` on 2026-08-11. What *was* measured here: none of the 18
|
||
patterns matches a legacy `sk-…T3BlbkFJ…` key shape. So a consumer vendoring this file
|
||
under-matches the seed hook by one entry, on a live credential shape, and now reads that in the
|
||
file rather than inferring it. It will be closed by re-extraction from a pinned public commit,
|
||
never by authoring the entry here from a message.
|
||
|
||
- `conformance/manifest.json` `0.3.0` → `0.3.1` — the `scope_planned.blockers` text for
|
||
`signatures/secret-egress.json` is corrected. Through `0.3.0` it ended by naming
|
||
`gcp-service-account-json` and `openai-api-key-legacy` together as ids "absent here". They are
|
||
two different kinds of fact, and one of them was misleading.
|
||
|
||
Measured 2026-08-11, by running this file's own 18 patterns in `order` over a service-account
|
||
document, against the guard at commit `e671edb`: a **complete** GCP service-account key file
|
||
*is* matched here, at order 11 (`Private Key PEM Block` — its `(?:RSA |EC |DSA |OPENSSH )?`
|
||
prefix group is optional, so the bare PKCS#8 header such a file carries matches). The same
|
||
document with `private_key` removed matches nothing here while the guard's marker pattern still
|
||
fires. That is a **cut-point** difference — the guard detects the document marker, this table
|
||
detects the key material — which is what the blocker is about, and not a missing entry.
|
||
`openai-api-key-legacy`, by contrast, is a real hole here today, and is now recorded as one.
|
||
|
||
The correction is folded into the existing blocker string rather than added as a sibling key:
|
||
`blockers` is a map from table path to text, and a second key under a table path would read as
|
||
a second table to anything iterating it.
|
||
|
||
- `docs/lexicon-port-divergence.md` (informative) — the residual `[^>]` vs `[^><]` row gains a
|
||
fuller witness set. `llm-security` measured the three forms as **totally ordered** by what they
|
||
match, each a strict superset of the next, and named two input classes the guard's narrower
|
||
class drops (`<script a="<" >x`, `<script<div>`) beyond the one commons had recorded.
|
||
Reproduced here independently, including the strict-superset property in both directions,
|
||
before being written down.
|
||
|
||
Their argument — that the narrower class buys an empty false-positive surface, since anything
|
||
reaching `[^>]`-and-not-`[^><]` already contains a literal `<script` tag — is recorded as
|
||
theirs and explicitly **not** what decided commons' form. Commons carries `[^>]` on provenance,
|
||
and would have carried `[^><]` had the source been the guard's. Also records that they asked to
|
||
hear the guard's reason for `[^><]` before commons shipped and commons shipped first, with why
|
||
that order was deliberate.
|
||
|
||
No data file touched; `v0.3.0` is unaffected.
|
||
|
||
## [0.3.0] — 2026-08-11
|
||
|
||
**A detection pattern changed value. That has not happened in this repository before, and it
|
||
is the reason this is a release.** `v0.2.0` changed what a runtime must *declare*; this one
|
||
changes what a conforming runtime *finds*. A consumer that vendors `0.3.0` and re-runs its
|
||
suite will see a finding on inputs that produced none under `0.2.0`. In 0.x that is a minor
|
||
bump by the rules; read the first entry below before upgrading, not the version number.
|
||
|
||
The lexicon `0.6.0` entry previously sitting under *Unreleased* is folded in here — it was
|
||
committed but never tagged, and `0.7.0` supersedes its central claim.
|
||
|
||
### Changed
|
||
|
||
- **`lexicon/injection-lexicon.json` (`0.5.1` → `0.7.0`) — `hybrid-xss:script-tag` converged
|
||
on `llm-security`'s current form.**
|
||
|
||
```
|
||
0.6.0 and earlier <script\b[^>]*>[\s\S]*?<\/script> closing tag REQUIRED
|
||
0.7.0 <script\b[^>]*> opening tag only
|
||
```
|
||
|
||
Byte-identical to `llm-security`'s `RegExp.prototype.source` at `b1ba1fb`
|
||
(`scanners/lib/injection-patterns.mjs:170`), verified by compiling both. They dropped the
|
||
closing-tag requirement in `90f576f` (2026-08-10) because it was a recall hole:
|
||
`<script>alert(1)` and `<script src=x.js>` both returned no finding.
|
||
|
||
**This is re-extraction, not revision, and the distinction is the whole justification.**
|
||
This repository's behaviour-preservation invariant forbids commons from *correcting* seed
|
||
data it believes is wrong — that rule stands and was not weakened. It does not forbid
|
||
re-reading the source after the source itself moved and its owner asked. The lexicon's
|
||
declared provenance is `llm-security`'s injection table, and being loadable verbatim by
|
||
that table's owner is the one thing it exists to do; the standing alternative was a
|
||
per-pattern override in `llm-security`'s own loader, i.e. a published core its source
|
||
repository could not load as published.
|
||
|
||
**Direction matters for what it cannot break:** the new form matches a strict superset of
|
||
the old one, so relative to `0.6.0` it can add matches and cannot introduce a false
|
||
negative. The reverse change would not have been adoptable on the same reasoning.
|
||
|
||
Operator decision, 2026-08-11, on `llm-security`'s blocking request. Explicitly **not**
|
||
decided by the 2-of-3 majority across the three ports: a count of implementations is not a
|
||
mandate over detection data, and the provenance argument would hold with the guard on
|
||
either side.
|
||
|
||
Measured collateral: **none.** The full corpus was run under both patterns — 84/84
|
||
lexicon-scoped cases pass under `0.7.0`, and exactly one case's finding set differs between
|
||
the two forms (the new one below). The widening added no finding to any other case's input.
|
||
|
||
`source_fidelity` restructured to keep its numbers coordinate-bearing:
|
||
`patterns_byte_identical_to_source` keeps its key and its value (83) and gains the field it
|
||
was missing, `byte_identical_against_commit: b1ba1fb`. Against the original extraction
|
||
commit `b0de0ca` this file is now 82/83, recorded as `count_against_extraction_commit`.
|
||
`post_extraction_drift` — added in the folded-in `0.6.0` to record the then-open divergence
|
||
— is now marked `status: resolved in 0.7.0 by re-extraction` and carries the before/after
|
||
pattern text, so a consumer diffing against either commit has a coordinate for what it
|
||
finds.
|
||
|
||
- **`conformance/manifest.json` (`0.2.0` → `0.3.0`) — `case_id_derivation` extended with an
|
||
optional variant suffix.**
|
||
|
||
```
|
||
before case_id = <pattern_id, ':' → '__'>
|
||
after case_id = <pattern_id, ':' → '__'> [ '--' <variant-slug> ]
|
||
reverse truncate at first '--', then '__' → ':'
|
||
```
|
||
|
||
No existing case id moved, so this is additive. `--` was measured absent from all 83
|
||
ratified pattern ids and all 89 pre-existing case ids, which keeps the reverse transform
|
||
purely lexical — no lookup against the id list — the property the original one-to-one rule
|
||
was protecting.
|
||
|
||
The `one_case_per_pattern_id` key is **removed**, superseded by
|
||
`case_id_derivation.variant_suffix.supersedes`, which quotes its text. It was documentation
|
||
of the constraint, not data a consumer matches on, but it is called out here because a
|
||
removed key is normally a breaking change in this repository.
|
||
|
||
`omitted_payloads` gains `derivation_ground_withdrawn_in_0_3_0`: the guard's seventh
|
||
active-content payload was omitted on two grounds, and this change retires one of them. The
|
||
other stands, so the payload stays omitted — on one ground instead of two. **It was not
|
||
added back**; that is a separate decision, not a consequence of this one.
|
||
|
||
### Added
|
||
|
||
- **`conformance/hybrid-xss__script-tag--src-no-close/` (89 → 90 cases)** — input
|
||
`<script src=x.js>`, 17 bytes, expecting `hybrid-xss:script-tag`. The regression gate for
|
||
the change above, and the reason the corpus could not previously see it: the existing
|
||
`hybrid-xss__script-tag` input `<script>steal()</script>` matches the pattern under *both*
|
||
forms, so it passes either way. Reverting the pattern to its `0.6.0` form fails this case
|
||
and only this case — mutation-verified in both directions across all 90.
|
||
|
||
**The first case input authored in this repository** rather than reproduced verbatim from a
|
||
runtime's payload set, recorded in the new `authored_payloads` block rather than folded into
|
||
`payload_provenance`, whose value is precisely the claim that its inputs are verbatim
|
||
upstream. That claim stays exactly as strong as it was: 83 of 83. Both witnesses for this
|
||
axis were named by `llm-security` on 2026-08-10; this is the first of the two. Findings
|
||
measured through the guard's public API at `0dce50f` / `0.5.0`, with the existing case's
|
||
committed bytes and digest reproduced by the same harness in the same run as a control.
|
||
|
||
- **`schema/conformance-declaration.schema.json` (`0.1.0`)** — the shape a runtime publishes
|
||
alongside a conformance result, satisfying the §1.1 MUST that `v0.2.0` created and left
|
||
without a form. Requested by `llm-security` in those terms (runtime, commit measured,
|
||
implemented file paths) with the stated reason that two runtimes publishing free-form
|
||
declarations makes `83/83 + 6 not-applicable` unparseable by anyone but its author.
|
||
|
||
Carries the two arithmetic invariants §1.1 implies but cannot state unambiguously in prose:
|
||
the four verdict counts MUST sum to the total, and the total MUST equal the corpus case
|
||
count at the commit measured. Requires the enumeration arrays whenever their counts are
|
||
non-zero, which turns §1.1's "MUST still be enumerated" from prose into a schema failure.
|
||
Keeps `error` and `not_applicable` structurally distinct, per §1.1. Records
|
||
`declaration_source` — whether the declared set is derived from the runner's own constant or
|
||
hand-maintained beside it — because only the derived form makes the anti-narrowing fence
|
||
structural. **Deliberately not a gate:** nothing in this repository runs, and no validation
|
||
was asked for. Mutation-tested: the example validates, and five distinct defect classes are
|
||
rejected.
|
||
|
||
- **`spec/conformance-corpus.md` §1.1** — normative pointer to that schema, plus a SHOULD that
|
||
a runtime derive its declared set from the constant its runner uses to accept or reject a
|
||
`scope`, and record which it did.
|
||
|
||
### Fixed
|
||
|
||
- `docs/lexicon-port-divergence.md` — the `hybrid-xss:script-tag` row is closed on the
|
||
closing-tag axis, having reversed twice in three days (guard-diverges → commons-diverges →
|
||
converged). What remains open is the one-byte span difference: the guard excludes `<` from
|
||
its negated class and the other two do not, so `<script <x>` matches commons and
|
||
`llm-security` and not the guard. Measured by compiling all three forms, not reasoned from
|
||
the character classes; neither side has claimed it.
|
||
|
||
## [0.2.0] — 2026-08-11
|
||
|
||
The contract gained a normative MUST, which is why this is a release rather than a
|
||
metadata commit: **a runtime that conformed to `v0.1.0` does not conform to this one until
|
||
it declares the set of commons data files it implements.** In 0.x that is a minor bump by
|
||
the rules, but it is breaking in substance, and a consumer reading only the version number
|
||
should learn that here rather than from a failing suite.
|
||
|
||
Everything below this heading was previously listed as unreleased.
|
||
|
||
### Retracted
|
||
|
||
- **The claim that the Python guard's port cites `severity.mjs` for hybrid severity.** It is
|
||
false. It was carried in three places — `lexicon/injection-lexicon.json`
|
||
(`families[hybrid].severity_provenance.not_from`), `docs/lexicon-port-divergence.md`
|
||
(§ *Severity: the 8 hybrid patterns*), and the `[0.1.0]` entry below — and it was never
|
||
measured here. It restated an assertion received from `llm-security` (coord message
|
||
`20260809T201048Z`) as a commons finding.
|
||
|
||
Measured against the guard's own tree, which `llm-ingestion-pipeline-security` asked for
|
||
twice before this was checked: `severity.mjs` has **never** appeared in
|
||
`src/llm_ingestion_guard/injection_lexicon.json` at any point in that file's history
|
||
(`git log -S` returns no commits), and at `0bf0729` — the commit
|
||
`conformance/manifest.json` pins — the only tree-wide occurrence is `docs/PLAN.md:114`,
|
||
correctly attributing the *report* module to `output.mjs` + `severity.mjs`. The guard's
|
||
only source statement for the lexicon is the `note` at `injection_lexicon.json:3`, and it
|
||
names `injection-patterns.mjs`.
|
||
|
||
**No detection data moves.** `families[hybrid].severity` is still `high`, still sourced to
|
||
`injection-patterns.mjs:274-281`, re-verified at `b0de0ca`; `severity.mjs` still contains
|
||
zero occurrences of `CRITICAL_PATTERNS`, `HIGH_PATTERNS`, `MEDIUM_PATTERNS` and
|
||
`HYBRID_PATTERNS`, re-measured the same day. Only the sentence about the *other* repository
|
||
falls.
|
||
|
||
The retraction is marked in place rather than edited away, and it is worth naming why this
|
||
one survived review: the claim arrived bundled with a correct measurement of the same
|
||
question, from a repository that had done its half properly. The correct half carried the
|
||
incorrect half past the check — which is precisely the defect
|
||
`severity_provenance.not_from` was written to warn about, one level up.
|
||
|
||
### Added
|
||
|
||
- **`not-applicable`, a third conformance verdict** (`spec/conformance-corpus.md` §1.1). A
|
||
runtime now declares the set of commons data files it implements; a case whose `scope`
|
||
names a file outside that set is `not-applicable` rather than failed. §1 alone would have
|
||
reported an architectural difference as a defect — one seeding runtime has no
|
||
active-content table and never will, and 7 permanent failures say nothing a reader can use.
|
||
The verdict is fenced so it cannot become an exit: it attaches to a **table**, never to a
|
||
case (per-case opt-out is the silent skip §1 forbids), a declared set MUST NOT be narrowed
|
||
to convert failures into `not-applicable`, and such cases MUST still be enumerated rather
|
||
than dropped from the denominator. §8 now states the consequence: a pass count is
|
||
unreadable without the declared set beside it.
|
||
|
||
- **Six active-content conformance cases** — `active__markdown-image`, `active__markdown-link`,
|
||
`active__reference-link`, `active__autolink`, `active__raw-html`, `active__data-uri`. The
|
||
corpus goes 83 → 89, and `scope_covered` gains `signatures/active-content.json`.
|
||
|
||
Generated from measurement, not written: payload strings were extracted from the seed
|
||
runtime's `coverage.py` by AST — evaluating each `_scan_case` argument in that module's own
|
||
namespace rather than retyping detection data — then run through its public output gate.
|
||
The fixtures were then re-read from disk by a separate checker that re-computed every
|
||
digest, re-scanned the bytes and applied `exact-within-scope` independently of the
|
||
generator, because a generator agreeing with itself proves nothing: 6 cases, 0 failed
|
||
checks.
|
||
|
||
**These six prove less than the 83, and the manifest says so.** Their payloads come from
|
||
the only runtime implementing the table, so no second implementation's agreement could be
|
||
measured. They pin one runtime's behaviour as a contract a future implementer can be held
|
||
to — which is less than cross-runtime agreement and more than nothing.
|
||
|
||
- `signatures/active-content.json` **0.1.0 → 0.2.0** — a `pattern_id_space` block. Unlike the
|
||
lexicon's, nothing was constructed: `label_format` and the `constructs` keys were already
|
||
extracted verbatim, and their concatenation *is* what the seed runtime emits. The block
|
||
states an id space the file already had implicitly, and records that it is ratified by
|
||
**one** runtime rather than two.
|
||
|
||
### Changed
|
||
|
||
- `lexicon/injection-lexicon.json` **0.5.0 → 0.5.1** — provenance metadata only; no pattern,
|
||
id, alias, family or severity value changes.
|
||
|
||
- `conformance/manifest.json` **0.1.1 → 0.2.0** — the six cases, `scope_covered`,
|
||
`count_by_scope`, separate provenance and measurement blocks for the active-content half
|
||
(a different source structure at a different commit; one pin must not stand for two
|
||
measurements), and `scope_planned.blockers`.
|
||
|
||
- **`spec/conformance-corpus.md` §4 no longer claims scoping "asks a question both can
|
||
answer."** That held only while every case was scoped to the one table both runtimes
|
||
implement, and stopped being true the moment a case was scoped to a single-runtime table.
|
||
Scope narrows *what* is compared; it does not make every runtime a valid addressee. The
|
||
superseded sentence is named in place rather than edited away.
|
||
|
||
### Fixed
|
||
|
||
- **§4 now states that "belongs to a data file" means published there, never "shares its
|
||
prefix."** The distinction has a live witness: the seed runtime emits `active:oversize-input`,
|
||
a self-safety flag about its own scan cap, which carries the `active:` prefix but is no
|
||
construct in `signatures/active-content.json`. A prefix-matching runtime would pull it into
|
||
the comparison and fail a case over a finding the corpus makes no claim about. Recorded in
|
||
that file under `pattern_id_space.not_every_active_label` as well.
|
||
|
||
- **§6 now states the derivation's cost.** `case_id` derives from `pattern_id` alone, so a
|
||
single-finding scope holds at most one case per pattern id — there is nowhere in the name
|
||
for a second. The seed runtime's matrix drives *two* payloads at `active:markdown-image`;
|
||
measured, their in-scope finding sets are identical, and the second's only distinguishing
|
||
signal (`entropy:base64-blob`) falls outside every table this repository publishes. It was
|
||
dropped rather than given a discriminated id, which would have broken the reverse
|
||
transform, and it is named in `conformance/manifest.json` under `omitted_payloads` so that
|
||
6 built from 7 offered reads as a decision rather than a miscount.
|
||
|
||
### Measured, not shipped
|
||
|
||
- **The remaining four cases are blocked on two distinct unresolved questions**, now recorded
|
||
under `scope_planned.blockers` instead of the earlier blanket "no runtime has agreed to an
|
||
id space". That framing was wrong for both:
|
||
|
||
- **Carriers (3).** No adoptable id space, and a second problem underneath. The guard emits
|
||
two *stage-coupled* labels for one carrier — `sanitize:zero-width` on input,
|
||
`output:zero-width-present` on output, same split for bidi and unicode-tag — while
|
||
llm-security emits prose titles. A commons id must be invented stage-neutral, which no
|
||
other id space here required. And since `exact-within-scope` compares a finding *set*, an
|
||
id aliasing both labels makes the verdict depend on which entry point the runtime was
|
||
measured through — an entry-point dependence the lexicon cases do not have.
|
||
- **Secret egress (1).** Not an id-naming question at all. The two runtimes carry
|
||
**different tables**: 18 entries here against the guard's 25, cut at different
|
||
granularities (this file's single `GitHub Token` is four ids there, `Private Key PEM
|
||
Block` three, `Database connection string` four), with membership diverging both ways.
|
||
`aws-access-key-id` is the one clean 1:1 — which is why exactly one egress case was ever
|
||
offered. That number was a symptom, not modesty. A shared id space presupposes a table
|
||
reconciliation nobody has done.
|
||
|
||
## [0.1.0] — 2026-08-10
|
||
|
||
Initial extraction. Runtime-neutral detection data, the finding contract, and a conformance
|
||
corpus, extracted from the `llm-security` Node implementation and a Python guard **without
|
||
behaviour change** — that invariant is the release, not a caveat on it.
|
||
|
||
What the tag is worth resting on: seven of the eight JSON artefacts were rebuilt from the
|
||
commons file alone and diffed against their source implementation, three of them against the
|
||
source module at a pinned commit. The eighth says `verified: false` about itself. The corpus
|
||
holds 83 cases on which both seeding runtimes were measured agreeing exactly.
|
||
|
||
What it is not: `spec/decode-pipeline.md` does not exist, and the corpus constrains one of
|
||
the seven data files. Both absences are named in *Not included* rather than papered over.
|
||
|
||
### Added
|
||
|
||
- `conformance/` — **83 cases, one per injection-lexicon pattern**, plus `manifest.json`.
|
||
Each case is a directory holding `input.txt` (the exact bytes, no trailing newline) and
|
||
`expected.json` (the findings, named by commons pattern `id`).
|
||
|
||
Both seeding runtimes were measured producing the **same lexicon finding set on all 83**,
|
||
through their public entry points — `scanForInjection()` at `b0de0ca` and
|
||
`scan_output(source=OUTPUT)` at `0bf0729` — with labels mapped to commons ids through the
|
||
lexicon's own `aliases` block. Not through rebuilt regex tables: a table-level comparison
|
||
yields a number that describes neither runtime, which is the mistake the divergence
|
||
document had to retract.
|
||
|
||
**The 13 divergent patterns are in, unmarked, and that is the substantive result.** Their
|
||
divergence was measured on witness inputs — an attribute run padded past 256 characters,
|
||
an interior `<`, an unclosed `<script>` — and none of those shapes occurs in a corpus
|
||
payload. All 13 agree on their own case input. Nobody had to pick whose recall cost
|
||
becomes the contract, because the question was never reachable from these inputs. A
|
||
per-case caveat would have asserted a doubt the measurement disproves.
|
||
|
||
Inputs are the guard's `coverage.py` payloads, reproduced verbatim. One runtime authored
|
||
them; what makes them a cross-runtime corpus is the measurement through the other, and the
|
||
manifest records the asymmetry rather than averaging it away.
|
||
|
||
- `spec/conformance-corpus.md` — **normative.** How a case is read: `input.txt` is bytes and
|
||
is not to be trimmed or re-encoded, `expected.json` names findings by `pattern_id` only
|
||
(severity and OWASP anchor are looked up in the lexicon, never restated), and
|
||
`exact-within-scope` requires equality **restricted to the data files the case names**.
|
||
|
||
The field is `pattern_id`, not `id`, because this repository already publishes an unrelated
|
||
finding `id`: `schema/finding.schema.json` defines it as `DS-<scanner>-<counter>` from a
|
||
process-global counter — stable across neither runs nor processes. Two normative documents
|
||
using one word for a stable rule identity and a volatile per-emission sequence number would
|
||
have produced runtimes failing every case for reasons unrelated to detection. §3.1 states
|
||
the distinction and publishes the bridge a runtime actually needs: its own label maps to a
|
||
`pattern_id` through the lexicon's `aliases` object, and a runtime absent from that object
|
||
has no published way to be compared at all.
|
||
|
||
Scoping is what makes exactness safe — the two runtimes do not implement the same set of
|
||
tables, so a whole-report comparison would fail for reasons unrelated to the pattern under
|
||
test. Exactness is what makes the corpus worth running — a contains-only corpus is passed
|
||
by a runtime that flags everything. `observed_out_of_scope` is evidence, never expectation,
|
||
and an absent runtime key means **unmeasured**, not measured-empty.
|
||
|
||
The document also states the one place this repository's "every JSON file carries a
|
||
top-level `version`" convention does not apply: fixtures are versioned as a corpus, in
|
||
`conformance/manifest.json`. Stated rather than left to be discovered.
|
||
|
||
- `schema/finding.schema.json` — the finding contract plus the SARIF output profile.
|
||
Normative. Closed against the producer in 0.2.0; the JSONL profile is `not applicable`.
|
||
- `signatures/active-content.json` — the EchoLeak class (CVE-2025-32711): 17 patterns,
|
||
severities, opacity floors and pass order, from the Python guard.
|
||
- `lexicon/injection-lexicon.json` — 83 prompt-injection patterns in four families
|
||
(21 critical, 32 high, 22 medium, 8 hybrid).
|
||
- `codepoints/carriers.json` — six carrier tables: zero-width characters, the Unicode Tags
|
||
block, the Supplementary Private Use Areas, BIDI controls, the Cyrillic presence set and
|
||
the 28-entry fold-to-Latin homoglyph map.
|
||
- `signatures/secret-egress.json` — the 18 fixed credential and token shapes. Array order
|
||
is normative.
|
||
- `mapping/owasp-map.json` — four taxonomy maps (LLM, ASI, AST, MCP) over one shared
|
||
16-prefix key set.
|
||
- `calibration/calibration.json` — risk-score tier constants, verdict thresholds, risk-band
|
||
cutoffs, posture grade thresholds.
|
||
- `signatures/malware-signatures.json` — the known-bad-identity table for the `SIG` class:
|
||
seven signatures over four families (`webshell`, `reverse_shell`, `cryptominer`,
|
||
`hacktool`), reproduced verbatim from `knowledge/signatures.json` at `b0de0ca`, key order
|
||
included, with the source file's byte length and SHA-256 pinned in `provenance`.
|
||
|
||
The rules were the easy half. The file's substance is the line between the table and the
|
||
engine, drawn in `engine_behaviour_not_data`: **no rule carries a `flags` field**, because
|
||
the engine compiles every pattern with `i` unconditionally — so a consumer that compiles
|
||
these case-sensitively silently under-matches all seven. Each pattern is also run against
|
||
five decode variants, not just raw bytes; rules are filtered by an enabled-families policy;
|
||
a rule fires once per file; operator rules are merged at scan time; and the loader defaults
|
||
four missing fields rather than rejecting a rule. None of that travels with the data, and
|
||
all of it changes what a consumer sees.
|
||
|
||
Two honesty notes are in `evidence_limits` rather than in prose. Seven signatures are not
|
||
malware coverage — a clean `SIG` result is not "no malware", and the seed runtime's own
|
||
header calls the table "deliberately tight". And three of the seven match on **names**
|
||
(`xmrig`, `mimikatz`, `meterpreter`), so a document *discussing* those tools matches; the
|
||
seed runtime papers over this by excluding `knowledge/`, `tests/`, `docs/` and
|
||
`node_modules/` from the scan, which is engine behaviour and does not come with the table.
|
||
|
||
Verified: 7/7 rule objects field-identical to source including key order, no non-ASCII
|
||
bytes, and all seven compile in Node bare, `i` and `iu` (21/21) and in Python `re` (7/7).
|
||
Note the exact family spellings — `reverse_shell`, not `reverse-shell`, and `cryptominer`,
|
||
not `miner`; they are policy keys, and the working note that seeded this file had both wrong.
|
||
|
||
### Verification
|
||
|
||
Every file above except `calibration.json` was proven rather than transcribed: the data was
|
||
rebuilt **from the commons JSON alone** and diffed against the source implementation. Each
|
||
file records its own result and its own limits.
|
||
|
||
`calibration/calibration.json` carries `verified: false`. Its source arrived as a prose
|
||
summary rather than as code, so no differential check was possible, and the file names the
|
||
checks that were not run instead of attaching a caveat to a pass.
|
||
|
||
The corpus was verified the same way the data was — by a harness that does **not** share the
|
||
generator's knowledge. It reads only the case directories, re-runs both runtimes on the bytes
|
||
it finds there, and checks every field of every `expected.json`, digests included: **83
|
||
cases, 0 failures**. Two further checks, because a corpus that cannot fail is not evidence:
|
||
commons' family severity matches the severity the guard emits per finding, **83/83**; and
|
||
deleting the middle third of each input breaks **76 of 83** expectations. The 7 survivors are
|
||
the shortest payloads, where the mutation leaves the trigger intact — that is a weak
|
||
mutation, not a weak fixture, and it is recorded as such rather than rounded up.
|
||
|
||
- `docs/lexicon-port-divergence.md` — informative. A differential comparison of the two
|
||
ports of `injection-patterns.mjs` (this repository's and the Python guard's): 83/83
|
||
patterns correspond, 64 are byte-identical, 6 differ only by escaping and are proven
|
||
equivalent, and **13 behave differently**, with a witness input for each and misses on
|
||
both sides. The cause is two different ReDoS mitigations of one table. **No data file was
|
||
changed** — behaviour preservation holds and the finding is reported to the owning
|
||
repositories.
|
||
|
||
**Revised 2026-08-09 with one retraction.** The document claimed that *neither runtime
|
||
misses an attack*, on the grounds that every witness payload still produced a finding. It
|
||
does miss. That measurement ran the payloads against the **union of every pattern table
|
||
this repository holds**, and the rescuing hit came from `active-content.json` — the Python
|
||
guard's table. `llm-security` has no active-content table at all, so a union of commons
|
||
tables was read as a statement about each runtime separately. Re-measured through
|
||
`llm-security`'s own `scanForInjection()` at `b0de0ca`, all three witness payloads return
|
||
**`found: false`** — no finding whatsoever — while controls in the same run behave
|
||
normally. Three confirmed recall holes, which `llm-security` attributes to its v7.8.3 #24
|
||
ReDoS hardening and has logged as a v8.x task.
|
||
|
||
Also corrected: one of the 13 divergences does not reach report level, because the guard's
|
||
`hybrid-xss:javascript-uri` fires on the same witness at the same severity and anchor. The
|
||
report-level number is **12**. And the `hybrid` severity question that the document reported
|
||
rather than resolved is now closed — the reported hint was right, the citation behind it was
|
||
not.
|
||
|
||
**Revised again 2026-08-10.** The document said 13 was the number blocking `conformance/`,
|
||
since a fixture names labels. It blocks a fixture written over a **witness** input, and the
|
||
corpus contains none — all 13 agree on their own case input. The divergence itself stands
|
||
unresolved and unchanged; what was wrong was the claim about what it blocked.
|
||
|
||
Corrections are marked in place rather than edited away.
|
||
|
||
### Changed
|
||
|
||
- `schema/finding.schema.json` **0.1.0 → 0.2.0** — the schema is **closed**. It was seeded
|
||
from `sarif-formatter.mjs`, which *consumes* findings, so its property list could only ever
|
||
be a lower bound and `additionalProperties` had to stay open. The producer is now known —
|
||
`finding()` in `scanners/lib/output.mjs`, line 32 — and it returns an object literal with
|
||
**exactly ten keys and no spread**: `id`, `scanner`, `severity`, `title`, `description`,
|
||
`file`, `line`, `evidence`, `owasp`, `recommendation`. `additionalProperties` is `false`,
|
||
and the two keys the old schema never knew about (`id`, `evidence`) are added.
|
||
|
||
`id` gets its own definition: `DS-<prefix>-<counter>`, pattern `^DS-[A-Za-z]+-[0-9]{3,}$`.
|
||
The `{3,}` is deliberate — `padStart(3, '0')` is a minimum, so a run emitting more than 999
|
||
findings produces four digits. The id comes from a process-global counter, so it is stable
|
||
neither across runs nor across processes, and the definition says so before someone keys on it.
|
||
|
||
Nullability is now evidence rather than convention. Five keys are emitted as `null` rather
|
||
than omitted (`opts.x || null`), so a serialised finding always carries all ten. The
|
||
exception is the four assigned straight from `opts`: omit `description` and the key is
|
||
`undefined` and vanishes from the JSON. Verified by calling the real producer — ten keys in
|
||
memory, nine after serialisation.
|
||
|
||
**`owasp` is a string, not an array, and not one code.** Multiple codes are joined with
|
||
`, `. Measured across the seed runtime: 31 distinct values over 157 emission sites, 13 of
|
||
them multi-code, and **four mix taxonomies inside a single value** (`LLM06, ASI02` and
|
||
friends) with no discriminator saying which is which. That sharpens the edition problem
|
||
`mapping/owasp-map.json` already records, and it has a consequence nobody had written down:
|
||
`sarif-formatter.mjs` builds `tags: [f.owasp]`, so a finding anchored to two taxonomies
|
||
produces **one** SARIF tag with a comma in it. Nothing filtering on `LLM06` will match.
|
||
Reproduced end to end through the real `finding()` and `toSARIF()`, and logged as
|
||
`known_lossiness.owasp-tag-not-split` — consumer behaviour in `llm-security`, not data, so
|
||
it is reported rather than fixed here.
|
||
|
||
The **JSONL profile is `not applicable`, not `unspecified`** — the distinction is the point.
|
||
`unspecified` would claim a profile exists and merely has not been written down. No
|
||
finding-JSONL exists: findings are emitted only inside a single JSON envelope
|
||
(`output.mjs:140`). The one module that does write JSONL, `audit-trail.mjs`, writes *audit
|
||
events* under a different schema — where `owasp` is an **array**. Same field name, different
|
||
type, same repository. A consumer reading both through one code path will be wrong about one
|
||
of them, so the profile records the trap instead of leaving a TODO.
|
||
|
||
Verified: the schema is valid Draft 2020-12, every finding built by the real producer
|
||
validates against it, and four negative controls (extra property, missing `id`, malformed
|
||
`id`, unknown severity) are all rejected.
|
||
|
||
One new open question, unpatched by design: the producer's JSDoc lists **seventeen** scanner
|
||
prefixes including `IDE`, while all four maps in `mapping/owasp-map.json` are keyed on
|
||
**sixteen** without it. An `IDE` finding has no taxonomy mapping in any map. Adding the key
|
||
would be inventing detection data.
|
||
|
||
- `lexicon/injection-lexicon.json` **0.4.0 → 0.5.0** — the last null in the file is filled and
|
||
the id space is ratified. Two blockers close, no detection data moves.
|
||
|
||
`families[hybrid].severity` was `null`, deliberately, because the seed dump did not supply
|
||
it. It is **`high`** — and the interesting part is where that is written. The hybrid family
|
||
has no severity field anywhere; the engine assigns one by pushing `HYBRID_PATTERNS` matches
|
||
straight into the `high` bucket at `injection-patterns.mjs:274-281`. Both this repository
|
||
and the Python guard had first looked in `severity.mjs`, which contains no injection-family
|
||
severity at all. The guard's port holds the right value behind that wrong citation, so
|
||
`severity_provenance.not_from` records the miss explicitly: a wrong citation to a right
|
||
value is the harder defect to catch later.
|
||
|
||
> **Correction 2026-08-10 (see Unreleased):** the two sentences about *the guard's* citation
|
||
> are false and were never measured here. The guard's port cites `injection-patterns.mjs`,
|
||
> the right file. Everything above about `severity.mjs` containing no injection-family
|
||
> severity, and about where the value actually lives, stands and has been re-measured.
|
||
|
||
`pattern_id_space.not_yet_confirmed` is replaced by `ratification`. Both seeding runtimes
|
||
agreed on 2026-08-09 — `llm-security` ratified the 0.2.0 proposal as-is and treats an id
|
||
change as breaking on the same terms, and the guard confirmed the space its own port
|
||
supplied. `id` is now a cross-runtime contract, which is what `conformance/` was waiting
|
||
on to be able to name a finding.
|
||
|
||
`alias_evidence.llm_security` is sharpened rather than upgraded. All 83 alias strings were
|
||
confirmed equal to the module's `label` field, in order — so the alias is certainly the
|
||
pattern's name **in the table**. It is still not established that a finding carries it: the
|
||
producer is `output.mjs:finding()`, which emits `title` and has no `label` key at all.
|
||
Verified at table level, one level short of where it would matter. Match on `id`.
|
||
|
||
- `lexicon/injection-lexicon.json` **0.3.0 → 0.4.0** — verified against the source module
|
||
instead of against the dump it was transcribed from, and **two false provenance claims
|
||
retracted**. The source is now pinned: `b0de0ca` on the public remote, imported in Node
|
||
and compared entry by entry on `source`, `flags` and `label`.
|
||
|
||
The result is **83/83 byte-identical to source**, which is not what the file previously
|
||
claimed. It said two patterns had been rewritten from raw code points into `\uXXXX`
|
||
escapes; the module already writes them escaped, so nothing had been rewritten. The stored
|
||
pattern text was right the whole time — only the account of where it came from was wrong.
|
||
The dump had rendered the module's escapes as the characters they denote, and this
|
||
repository re-escaped them, arriving at the correct bytes by way of an incorrect story.
|
||
|
||
The same inversion ran the other way in `multi-lang:french`, which carried the class
|
||
spelled with a raw accented Latin `e` where the module writes it as the escape
|
||
`\u00e9` inside the same character class.
|
||
That was the one pattern of 83 not byte-identical to source,
|
||
and it is corrected. The two spellings are the same regular expression — verified in Node
|
||
bare and under `u`, and in Python `re`, over accented, unaccented, uppercase and
|
||
non-matching French input, with identical match offsets — so **no behaviour moved**. No
|
||
pattern in the file contains a non-ASCII byte now, matching the module, whose regex
|
||
literals are pure ASCII throughout.
|
||
|
||
Structurally: `normalisations` is now `[]` with a `normalisations_note`, matching the
|
||
convention already used in `signatures/secret-egress.json`, and a new `source_fidelity`
|
||
block carries the counts, the method, the verified class membership, and both retractions
|
||
in full. Retracted claims are recorded rather than deleted — the earlier equivalence
|
||
evidence (692 Node comparisons, 236 Python) remains true, it is simply no longer
|
||
load-bearing.
|
||
|
||
- `lexicon/injection-lexicon.json` **0.2.0 → 0.3.0** — the two aliases are no longer presented
|
||
as equally backed. `pattern_id_space.alias_evidence` now records each one separately:
|
||
`llm_ingestion_guard` is **verified** (the guard's coverage matrix asserts on that exact
|
||
string, so it is demonstrably what a guard finding carries), while `llm_security` is
|
||
**not** — it is the pattern table's own name, and the finding producer was never supplied,
|
||
with the known Node finding shape using `title` rather than `label`. Averaging the two into
|
||
one file-level claim would have repeated the defect this repository corrects per-table
|
||
elsewhere.
|
||
|
||
Also: `normalisations[].affects` now keys on `id` with the prose names kept beside it as
|
||
`affects_labels`. An internal cross-reference on label was a second identity space inside
|
||
the file the id was added to unify.
|
||
|
||
- `lexicon/injection-lexicon.json` **0.1.0 → 0.2.0** — every pattern gains a commons-owned
|
||
`id` and an `aliases` object naming what each seeding runtime calls it, plus a top-level
|
||
`pattern_id_space` block explaining the field. This exists because a `conformance/`
|
||
fixture has to name a finding and the two runtimes do not name the same pattern the same
|
||
way.
|
||
|
||
The id was **adopted verbatim from the guard's port**, which already carried both names,
|
||
rather than invented here. Matching was by `label` ↔ `desc` with em-dash normalised to
|
||
hyphen: 83/83, one-to-one, ids unique.
|
||
|
||
**No detection data moved.** Labels, patterns and flags are byte-identical in sequence,
|
||
no `flags` key was invented (78 before, 78 after), and stripping the three new fields
|
||
reproduces the previous committed file byte for byte — 23 566 bytes, identical. All 83
|
||
patterns still compile in Node bare and under `u` (166/166) and in Python `re` (83/83).
|
||
|
||
Neither `llm-security` nor the guard has ratified this id space yet; both were asked by
|
||
coord on 2026-08-09, and the file says so rather than implying agreement.
|
||
|
||
### Not included
|
||
|
||
- `spec/decode-pipeline.md` — needs the decode implementation. A normative spec inferred
|
||
from a data dump would be worse than an absent one.
|
||
- **Conformance for the other four tables.** The corpus covers the injection lexicon only.
|
||
The carrier, active-content and secret-egress tables have 11 convertible cases waiting in
|
||
the guard's matrix, and no ratified cross-runtime finding id between them — writing those
|
||
fixtures would mint a contract unilaterally, in the same stroke as the tag. Named in
|
||
`conformance/manifest.json` under `scope_planned`.
|
||
- The 29 non-convertible cases of the guard's 134 assert a runtime's **API surface** — that
|
||
a Python call raises `OKFPathError`, that a disposition engine composes two findings a
|
||
particular way. This repository does not own an API, so those belong to the guard's suite.
|
||
|
||
`spec/decode-pipeline.md` is named in the README as planned rather than linked, so nothing
|
||
in the repository points at a file that does not exist.
|