fix(lexicon,docs): retract the claim that the guard's port cites severity.mjs

The guard asked twice which file:line carried it. Tracing it produced an
uncomfortable answer: no file of ours measured it. The claim arrived from
llm-security (coord 20260809T201048Z) and was written down here as a commons
finding in three places.

Measured against the guard's own tree: severity.mjs has never appeared in
src/llm_ingestion_guard/injection_lexicon.json at any point in that file's
history (git log -S: no commits), and at 0bf0729 - the commit our manifest
pins - the only tree-wide occurrence is docs/PLAN.md:114, correctly about the
report module. Their only lexicon source statement is the note at line 3, and
it names injection-patterns.mjs. The right file.

No detection data moves. hybrid severity is still high, still sourced to
injection-patterns.mjs:274-281 at b0de0ca; severity.mjs still has zero
occurrences of the four pattern arrays, re-measured today. Only the sentence
about the other repository falls.

Marked in place, not edited away. The claim survived review because it came
bundled with a correct measurement of the same question - a wrong citation to
a right value, which is exactly what the field it lived in was written to warn
against.

lexicon/injection-lexicon.json 0.5.0 -> 0.5.1 (provenance metadata only).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWKkmKQBsWtrkod4MusUBC
This commit is contained in:
Kjell Tore Guttormsen 2026-08-10 20:37:53 +02:00
commit f082a91995
3 changed files with 68 additions and 8 deletions

View file

@ -9,6 +9,43 @@ Versioning note: the repository tag versions **the contract** (file set, key nam
case ids, disposition semantics). Each JSON file additionally carries its own case ids, disposition semantics). Each JSON file additionally carries its own
`"version"` field, bumped when that file changes. `"version"` field, bumped when that file changes.
## [Unreleased]
### Retracted
- **The claim that the Python guard's port cites `severity.mjs` for hybrid severity.** It is
false. It was carried in three places — `lexicon/injection-lexicon.json`
(`families[hybrid].severity_provenance.not_from`), `docs/lexicon-port-divergence.md`
*Severity: the 8 hybrid patterns*), and the `[0.1.0]` entry below — and it was never
measured here. It restated an assertion received from `llm-security` (coord message
`20260809T201048Z`) as a commons finding.
Measured against the guard's own tree, which `llm-ingestion-pipeline-security` asked for
twice before this was checked: `severity.mjs` has **never** appeared in
`src/llm_ingestion_guard/injection_lexicon.json` at any point in that file's history
(`git log -S` returns no commits), and at `0bf0729` — the commit
`conformance/manifest.json` pins — the only tree-wide occurrence is `docs/PLAN.md:114`,
correctly attributing the *report* module to `output.mjs` + `severity.mjs`. The guard's
only source statement for the lexicon is the `note` at `injection_lexicon.json:3`, and it
names `injection-patterns.mjs`.
**No detection data moves.** `families[hybrid].severity` is still `high`, still sourced to
`injection-patterns.mjs:274-281`, re-verified at `b0de0ca`; `severity.mjs` still contains
zero occurrences of `CRITICAL_PATTERNS`, `HIGH_PATTERNS`, `MEDIUM_PATTERNS` and
`HYBRID_PATTERNS`, re-measured the same day. Only the sentence about the *other* repository
falls.
The retraction is marked in place rather than edited away, and it is worth naming why this
one survived review: the claim arrived bundled with a correct measurement of the same
question, from a repository that had done its half properly. The correct half carried the
incorrect half past the check — which is precisely the defect
`severity_provenance.not_from` was written to warn about, one level up.
### Changed
- `lexicon/injection-lexicon.json` **0.5.0 → 0.5.1** — provenance metadata only; no pattern,
id, alias, family or severity value changes.
## [0.1.0] — 2026-08-10 ## [0.1.0] — 2026-08-10
Initial extraction. Runtime-neutral detection data, the finding contract, and a conformance Initial extraction. Runtime-neutral detection data, the finding contract, and a conformance
@ -224,6 +261,11 @@ mutation, not a weak fixture, and it is recorded as such rather than rounded up.
`severity_provenance.not_from` records the miss explicitly: a wrong citation to a right `severity_provenance.not_from` records the miss explicitly: a wrong citation to a right
value is the harder defect to catch later. value is the harder defect to catch later.
> **Correction 2026-08-10 (see Unreleased):** the two sentences about *the guard's* citation
> are false and were never measured here. The guard's port cites `injection-patterns.mjs`,
> the right file. Everything above about `severity.mjs` containing no injection-family
> severity, and about where the value actually lives, stands and has been re-measured.
`pattern_id_space.not_yet_confirmed` is replaced by `ratification`. Both seeding runtimes `pattern_id_space.not_yet_confirmed` is replaced by `ratification`. Both seeding runtimes
agreed on 2026-08-09 — `llm-security` ratified the 0.2.0 proposal as-is and treats an id agreed on 2026-08-09 — `llm-security` ratified the 0.2.0 proposal as-is and treats an id
change as breaking on the same terms, and the guard confirmed the space its own port change as breaking on the same terms, and the guard confirmed the space its own port

View file

@ -166,12 +166,29 @@ differences in the table above are closed.
The part worth keeping is where the value lives. It is not a field. The engine assigns it by The part worth keeping is where the value lives. It is not a field. The engine assigns it by
pushing `HYBRID_PATTERNS` matches straight into the `high` bucket at pushing `HYBRID_PATTERNS` matches straight into the `high` bucket at
`injection-patterns.mjs:274-281`. The guard's port cites `severity.mjs` — a file that `injection-patterns.mjs:274-281`. `severity.mjs` contains **no injection-family severity at
contains **no injection-family severity at all**. So the guard held the right value behind a all** — re-measured 2026-08-10 at `b0de0ca`: `CRITICAL_PATTERNS`, `HIGH_PATTERNS`,
citation that leads nowhere, and a reviewer following that citation to check the number would `MEDIUM_PATTERNS` and `HYBRID_PATTERNS` appear there zero times.
have found nothing and drawn no conclusion. Refusing to copy it was the right call for a
reason better than the one given at the time: not merely that a port is second-hand, but that ~~**The guard's port cites `severity.mjs`.** So the guard held the right value behind a
this particular port could not have read what it claimed to. citation that leads nowhere, and refusing to copy it was right for a better reason than the
one given at the time: this particular port could not have read what it claimed to.~~
**Retracted 2026-08-10. The guard's port cites the right file.** This paragraph was never
measured here; it restated an assertion received from `llm-security`
(`20260809T201048Z`: *"Guardens port satte riktig verdi, men kunne ikke ha lest den fra fila
den oppgir"*) as a commons finding. Measured against the guard's own tree: `severity.mjs`
has **never** appeared in `src/llm_ingestion_guard/injection_lexicon.json` at any point in
that file's history (`git log -S` returns no commits), and at `0bf0729` — the commit
`conformance/manifest.json` pins — the only tree-wide occurrence is `docs/PLAN.md:114`,
correctly attributing the *report* module to `output.mjs` + `severity.mjs`. The guard's only
source statement for the lexicon is the `note` at `injection_lexicon.json:3`, and it names
`injection-patterns.mjs`.
Refusing to copy the value was still the right call — but for the plain reason, that a port
is second-hand evidence, not for the sharper one claimed above. The sharper reason was itself
a wrong citation to a right value, which is the defect this section was written to warn
about. It survived here because it arrived from a repository that had measured the *other*
half of the claim correctly, and the correct half carried the incorrect half past review.
## What this does not show ## What this does not show

View file

@ -1,5 +1,5 @@
{ {
"version": "0.5.0", "version": "0.5.1",
"id": "injection-lexicon", "id": "injection-lexicon",
"description": "Prompt-injection pattern lexicon: the four pattern families a detector matches against untrusted text, and the severity family each belongs to. Data only. The variant building that feeds these patterns (normalisation, homoglyph folding, rot13, unicode-tag escalation) and the buried-payload heuristic are engine behaviour and are deliberately NOT described here.", "description": "Prompt-injection pattern lexicon: the four pattern families a detector matches against untrusted text, and the severity family each belongs to. Data only. The variant building that feeds these patterns (normalisation, homoglyph folding, rot13, unicode-tag escalation) and the buried-payload heuristic are engine behaviour and are deliberately NOT described here.",
"owasp": "LLM01", "owasp": "LLM01",
@ -894,7 +894,8 @@
"source_lines": "274-281", "source_lines": "274-281",
"source_commit": "b0de0ca6d86ce697f39669d177c2c2654c280128", "source_commit": "b0de0ca6d86ce697f39669d177c2c2654c280128",
"verified": "directly, by reading the module at the pinned commit", "verified": "directly, by reading the module at the pinned commit",
"not_from": "scanners/lib/severity.mjs, which was the first file asked about and which contains no injection-family severity at all - grep for CRITICAL_PATTERNS, HIGH_PATTERNS, MEDIUM_PATTERNS or HYBRID_PATTERNS there returns nothing. The Python guard's port carries the correct value but cites that file, so a consumer following its citation would find no answer. Recorded because a wrong citation to a right value is the harder defect to notice." "not_from": "scanners/lib/severity.mjs, which was the first file asked about and which contains no injection-family severity at all - grep for CRITICAL_PATTERNS, HIGH_PATTERNS, MEDIUM_PATTERNS or HYBRID_PATTERNS there returns nothing at the pinned commit. That part stands and was re-measured 2026-08-10.",
"retracted_2026-08-10": "This field also said: 'The Python guard's port carries the correct value but cites that file, so a consumer following its citation would find no answer.' RETRACTED - it is false, and it was never measured here. It restated an assertion received from llm-security (coord message 20260809T201048Z) as if it were a commons finding. Measured against the guard's own tree: severity.mjs has never appeared in src/llm_ingestion_guard/injection_lexicon.json at any point in that file's history (git log -S returns no commits), and at 0bf0729 - the commit conformance/manifest.json pins - the only tree-wide occurrence is docs/PLAN.md:114, which correctly attributes the report module to output.mjs + severity.mjs and says nothing about injection-family severity. The guard's only source statement for the lexicon is the note field at injection_lexicon.json:3, and it cites injection-patterns.mjs - the right file. Kept rather than deleted because this repository's stated reason for recording the original claim was that a wrong citation to a right value is the harder defect to notice, and that turned out to describe this record itself."
}, },
"count": 8, "count": 8,
"patterns": [ "patterns": [