open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

release(0.3.0): a detection pattern changed value — that is new here

Browse source 
v0.2.0 changed what a runtime must DECLARE. This one changes what a conforming
runtime FINDS. A consumer that vendors 0.3.0 and re-runs its suite will see a
finding on inputs that produced none under 0.2.0. In 0.x that is a minor bump by
the rules; the CHANGELOG says so at the top rather than leaving it to be found in
a diff.

The lexicon 0.6.0 entry that sat under [Unreleased] is folded into this release —
committed but never tagged, and 0.7.0 supersedes its central claim (that commons
does not follow the upstream change).

Version sync: README's subtree-add and clone examples move v0.2.0 -> v0.3.0, the
two "as of v0.2.0" markers move with them, the corpus row goes 89 -> 90 cases and
83 -> 84 lexicon cases, and the scope breakdown now says 84 cases over 83 patterns
rather than implying one case per pattern. schema/conformance-declaration.schema.json
gains its own row; the corpus-spec row now mentions the declared table set, since
§1.1 is the reason a reader would go looking.

Verification run before tagging: all JSON well-formed; every non-conformance JSON
carries a top-level version; every spec carries Status: normative; charter guard
clean (no executable code); manifest count == cases[] length == directories on disk
== 90, no duplicate ids; every case's recorded bytes and sha256 recomputed from
disk, zero mismatches, no trailing newlines; measured count_by_scope equals the
manifest's; declaration schema valid draft 2020-12 with its example validating.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HuudQLGMyMenuyeZk8fKs5
This commit is contained in:
Kjell Tore Guttormsen 2026-08-11 13:39:51 +02:00
commit 4641a7b518
2 changed files with 137 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

138
CHANGELOG.md
View file

@ -11,21 +11,135 @@ case ids, disposition semantics). Each JSON file additionally carries its own
## [Unreleased] ## [Unreleased]
Nothing yet.
## [0.3.0] — 2026-08-11
**A detection pattern changed value. That has not happened in this repository before, and it
is the reason this is a release.** `v0.2.0` changed what a runtime must *declare*; this one
changes what a conforming runtime *finds*. A consumer that vendors `0.3.0` and re-runs its
suite will see a finding on inputs that produced none under `0.2.0`. In 0.x that is a minor
bump by the rules; read the first entry below before upgrading, not the version number.
The lexicon `0.6.0` entry previously sitting under *Unreleased* is folded in here — it was
committed but never tagged, and `0.7.0` supersedes its central claim.
### Changed
- **`lexicon/injection-lexicon.json` (`0.5.1` → `0.7.0`) — `hybrid-xss:script-tag` converged
on `llm-security`'s current form.**
```
0.6.0 and earlier <script\b[^>]*>[\s\S]*?<\/script> closing tag REQUIRED
0.7.0 <script\b[^>]*> opening tag only
```
Byte-identical to `llm-security`'s `RegExp.prototype.source` at `b1ba1fb`
(`scanners/lib/injection-patterns.mjs:170`), verified by compiling both. They dropped the
closing-tag requirement in `90f576f` (2026-08-10) because it was a recall hole:
`<script>alert(1)` and `<script src=x.js>` both returned no finding.
**This is re-extraction, not revision, and the distinction is the whole justification.**
This repository's behaviour-preservation invariant forbids commons from *correcting* seed
data it believes is wrong — that rule stands and was not weakened. It does not forbid
re-reading the source after the source itself moved and its owner asked. The lexicon's
declared provenance is `llm-security`'s injection table, and being loadable verbatim by
that table's owner is the one thing it exists to do; the standing alternative was a
per-pattern override in `llm-security`'s own loader, i.e. a published core its source
repository could not load as published.
**Direction matters for what it cannot break:** the new form matches a strict superset of
the old one, so relative to `0.6.0` it can add matches and cannot introduce a false
negative. The reverse change would not have been adoptable on the same reasoning.
Operator decision, 2026-08-11, on `llm-security`'s blocking request. Explicitly **not**
decided by the 2-of-3 majority across the three ports: a count of implementations is not a
mandate over detection data, and the provenance argument would hold with the guard on
either side.
Measured collateral: **none.** The full corpus was run under both patterns — 84/84
lexicon-scoped cases pass under `0.7.0`, and exactly one case's finding set differs between
the two forms (the new one below). The widening added no finding to any other case's input.
`source_fidelity` restructured to keep its numbers coordinate-bearing:
`patterns_byte_identical_to_source` keeps its key and its value (83) and gains the field it
was missing, `byte_identical_against_commit: b1ba1fb`. Against the original extraction
commit `b0de0ca` this file is now 82/83, recorded as `count_against_extraction_commit`.
`post_extraction_drift` — added in the folded-in `0.6.0` to record the then-open divergence
— is now marked `status: resolved in 0.7.0 by re-extraction` and carries the before/after
pattern text, so a consumer diffing against either commit has a coordinate for what it
finds.
- **`conformance/manifest.json` (`0.2.0` → `0.3.0`) — `case_id_derivation` extended with an
optional variant suffix.**
```
before case_id = <pattern_id, ':' → '__'>
after case_id = <pattern_id, ':' → '__'> [ '--' <variant-slug> ]
reverse truncate at first '--', then '__' → ':'
```
No existing case id moved, so this is additive. `--` was measured absent from all 83
ratified pattern ids and all 89 pre-existing case ids, which keeps the reverse transform
purely lexical — no lookup against the id list — the property the original one-to-one rule
was protecting.
The `one_case_per_pattern_id` key is **removed**, superseded by
`case_id_derivation.variant_suffix.supersedes`, which quotes its text. It was documentation
of the constraint, not data a consumer matches on, but it is called out here because a
removed key is normally a breaking change in this repository.
`omitted_payloads` gains `derivation_ground_withdrawn_in_0_3_0`: the guard's seventh
active-content payload was omitted on two grounds, and this change retires one of them. The
other stands, so the payload stays omitted — on one ground instead of two. **It was not
added back**; that is a separate decision, not a consequence of this one.
### Added ### Added
- `lexicon/injection-lexicon.json` (`0.5.1` → `0.6.0`) — `source_fidelity.post_extraction_drift`, - **`conformance/hybrid-xss__script-tag--src-no-close/` (89 → 90 cases)** — input
recording that `llm-security` commit `90f576f` (2026-08-10) dropped the closing-tag `<script src=x.js>`, 17 bytes, expecting `hybrid-xss:script-tag`. The regression gate for
requirement from `hybrid-xss:script-tag`'s source pattern, which this file — a frozen the change above, and the reason the corpus could not previously see it: the existing
extraction under behaviour-preservation — does not follow. `patterns_byte_identical_to_source` `hybrid-xss__script-tag` input `<script>steal()</script>` matches the pattern under *both*
(83, at extraction commit `b0de0ca`) is unchanged and still true; the new field adds the count forms, so it passes either way. Reverting the pattern to its `0.6.0` form fails this case
against `llm-security`'s later HEAD (82/83, measured against `b1ba1fb`, 2026-08-11) so a and only this case — mutation-verified in both directions across all 90.
consumer diffing against current upstream has a coordinate to resolve the mismatch against
instead of finding it unexplained. No pattern text, id, or existing key changed — additive
only, hence minor. Companion informative update in `docs/lexicon-port-divergence.md`, whose
`hybrid-xss:script-tag` row is reversed: commons is now the sole diverger of the three runtimes
on this one construct, not the guard.
Repository tag unaffected — no file set, key name, case id, or disposition semantics moved. **The first case input authored in this repository** rather than reproduced verbatim from a
runtime's payload set, recorded in the new `authored_payloads` block rather than folded into
`payload_provenance`, whose value is precisely the claim that its inputs are verbatim
upstream. That claim stays exactly as strong as it was: 83 of 83. Both witnesses for this
axis were named by `llm-security` on 2026-08-10; this is the first of the two. Findings
measured through the guard's public API at `0dce50f` / `0.5.0`, with the existing case's
committed bytes and digest reproduced by the same harness in the same run as a control.
- **`schema/conformance-declaration.schema.json` (`0.1.0`)** — the shape a runtime publishes
alongside a conformance result, satisfying the §1.1 MUST that `v0.2.0` created and left
without a form. Requested by `llm-security` in those terms (runtime, commit measured,
implemented file paths) with the stated reason that two runtimes publishing free-form
declarations makes `83/83 + 6 not-applicable` unparseable by anyone but its author.
Carries the two arithmetic invariants §1.1 implies but cannot state unambiguously in prose:
the four verdict counts MUST sum to the total, and the total MUST equal the corpus case
count at the commit measured. Requires the enumeration arrays whenever their counts are
non-zero, which turns §1.1's "MUST still be enumerated" from prose into a schema failure.
Keeps `error` and `not_applicable` structurally distinct, per §1.1. Records
`declaration_source` — whether the declared set is derived from the runner's own constant or
hand-maintained beside it — because only the derived form makes the anti-narrowing fence
structural. **Deliberately not a gate:** nothing in this repository runs, and no validation
was asked for. Mutation-tested: the example validates, and five distinct defect classes are
rejected.
- **`spec/conformance-corpus.md` §1.1** — normative pointer to that schema, plus a SHOULD that
a runtime derive its declared set from the constant its runner uses to accept or reject a
`scope`, and record which it did.
### Fixed
- `docs/lexicon-port-divergence.md` — the `hybrid-xss:script-tag` row is closed on the
closing-tag axis, having reversed twice in three days (guard-diverges → commons-diverges →
converged). What remains open is the one-byte span difference: the guard excludes `<` from
its negated class and the other two do not, so `<script <x>` matches commons and
`llm-security` and not the guard. Measured by compiling all three forms, not reasoned from
the character classes; neither side has claimed it.
## [0.2.0] — 2026-08-11 ## [0.2.0] — 2026-08-11

18
README.md
View file

@ -24,7 +24,7 @@ As a `git subtree` (recommended: history is preserved and upgrades are a single
```bash ```bash
git subtree add --prefix vendor/commons \ git subtree add --prefix vendor/commons \
https://git.fromaitochitta.com/open/llm-security-commons.git v0.2.0 --squash https://git.fromaitochitta.com/open/llm-security-commons.git v0.3.0 --squash
# later, to move to a newer tag # later, to move to a newer tag
git subtree pull --prefix vendor/commons \ git subtree pull --prefix vendor/commons \
@ -34,7 +34,7 @@ git subtree pull --prefix vendor/commons \
Or pin a tag and copy — `fork-and-own` is an explicitly supported path: Or pin a tag and copy — `fork-and-own` is an explicitly supported path:
```bash ```bash
git clone --depth 1 --branch v0.2.0 \ git clone --depth 1 --branch v0.3.0 \
https://git.fromaitochitta.com/open/llm-security-commons.git https://git.fromaitochitta.com/open/llm-security-commons.git
``` ```
@ -58,16 +58,17 @@ and keeping it that small is the point.
| [`calibration/calibration.json`](calibration/calibration.json) | The numbers a detector must not invent: risk-score tier constants, verdict thresholds, risk-band cutoffs, posture grade thresholds. Transcribed from a prose summary, not differentially verified — the file says so itself. | | [`calibration/calibration.json`](calibration/calibration.json) | The numbers a detector must not invent: risk-score tier constants, verdict thresholds, risk-band cutoffs, posture grade thresholds. Transcribed from a prose summary, not differentially verified — the file says so itself. |
| [`mapping/owasp-map.json`](mapping/owasp-map.json) | Finding-id prefix → OWASP taxonomy entry (LLM / ASI / AST / MCP). | | [`mapping/owasp-map.json`](mapping/owasp-map.json) | Finding-id prefix → OWASP taxonomy entry (LLM / ASI / AST / MCP). |
| [`schema/finding.schema.json`](schema/finding.schema.json) | **Normative.** The finding contract — closed against its producer, ten properties — plus the SARIF output profile. The JSONL profile is recorded as `not applicable`, with the reason. | | [`schema/finding.schema.json`](schema/finding.schema.json) | **Normative.** The finding contract — closed against its producer, ten properties — plus the SARIF output profile. The JSONL profile is recorded as `not applicable`, with the reason. |
| [`spec/conformance-corpus.md`](spec/conformance-corpus.md) | **Normative.** How to read the corpus: what a case is, why `input.txt` is bytes rather than text, and what `exact-within-scope` requires of a runtime. | | [`schema/conformance-declaration.schema.json`](schema/conformance-declaration.schema.json) | **Normative.** The shape a runtime publishes alongside a conformance result: which commons tables it implements, the commons commit it measured, and the four verdict counts. Required by the corpus spec §1.1; not validated by anything here, because nothing here runs. |
| [`conformance/`](conformance/) | 89 cases. One directory per case: `input.txt` in, `expected.json` out. Ground truth. 83 cover the injection lexicon, one per pattern, and both seeding runtimes were measured producing the same verdict on all 83. Six cover active content and are measured against the one runtime that implements that table — `not-applicable` for the other, not failing. See [`conformance/manifest.json`](conformance/manifest.json). | | [`spec/conformance-corpus.md`](spec/conformance-corpus.md) | **Normative.** How to read the corpus: what a case is, why `input.txt` is bytes rather than text, what `exact-within-scope` requires of a runtime, and how a runtime declares its table set so a case scoped outside it reads as `not-applicable` rather than as a failure. |
| `spec/decode-pipeline.md` | **Planned, still not shipped as of v0.2.0.** The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input. Writing it needs the decode implementation, which is engine code and has not been supplied — and a normative spec guessed from a data dump would be worse than an absent one. | | [`conformance/`](conformance/) | 90 cases. One directory per case: `input.txt` in, `expected.json` out. Ground truth. 84 cover the injection lexicon — 83 one per pattern, both seeding runtimes measured producing the same verdict on all 83, plus one variant case gating a pattern form against its predecessor. Six cover active content and are measured against the one runtime that implements that table — `not-applicable` for the other, not failing. See [`conformance/manifest.json`](conformance/manifest.json). |
| `spec/decode-pipeline.md` | **Planned, still not shipped as of v0.3.0.** The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input. Writing it needs the decode implementation, which is engine code and has not been supplied — and a normative spec guessed from a data dump would be worse than an absent one. |
| [`docs/extraction-plan.md`](docs/extraction-plan.md) | Informative: where each file was seeded from, and what v0.1.0 promised. | | [`docs/extraction-plan.md`](docs/extraction-plan.md) | Informative: where each file was seeded from, and what v0.1.0 promised. |
| [`docs/lexicon-port-divergence.md`](docs/lexicon-port-divergence.md) | Informative: a measured disagreement between two ports of the injection lexicon — 13 patterns that behave differently, in both directions, and why no data file was changed because of it. | | [`docs/lexicon-port-divergence.md`](docs/lexicon-port-divergence.md) | Informative: a measured disagreement between two ports of the injection lexicon — 13 patterns that behave differently, in both directions, and why no data file was changed because of it. |
Every JSON file carries a top-level `version`. Every normative specification carries a Every JSON file carries a top-level `version`. Every normative specification carries a
`Status: normative` marker. Rows marked **Planned** are named here because the layout is `Status: normative` marker. Rows marked **Planned** are named here because the layout is
part of the contract, but the file does not exist yet — they are not links, and nothing in part of the contract, but the file does not exist yet — they are not links, and nothing in
v0.2.0 depends on them. v0.3.0 depends on them.
Each data file records its own provenance and, in `verified`, how strongly it is backed. Each data file records its own provenance and, in `verified`, how strongly it is backed.
`calibration/calibration.json` is currently the one file that says `false`: it was `calibration/calibration.json` is currently the one file that says `false`: it was
@ -91,8 +92,9 @@ runs in each consumer's own test suite, against a pinned tag.
The corpus covers two tables, and they do not carry equal weight — treating them as one The corpus covers two tables, and they do not carry equal weight — treating them as one
number would misreport both: number would misreport both:
- `lexicon/injection-lexicon.json` — 83 cases. Both seeding runtimes implement it and both - `lexicon/injection-lexicon.json` — 84 cases over 83 patterns. Both seeding runtimes
ratified its id space. implement it and both ratified its id space. One pattern carries a second, variant case;
see `case_id_derivation.variant_suffix` in the manifest.
- `signatures/active-content.json` — 6 cases. One runtime implements it. For a runtime that - `signatures/active-content.json` — 6 cases. One runtime implements it. For a runtime that
does not, these cases are **`not-applicable`**, a third verdict beside pass and fail: a does not, these cases are **`not-applicable`**, a third verdict beside pass and fail: a
runtime declares which commons data files it implements, and a case scoped outside that runtime declares which commons data files it implements, and a case scoped outside that