fix(lexicon,docs): retract the claim that the guard's port cites severity.mjs
The guard asked twice which file:line carried it. Tracing it produced an uncomfortable answer: no file of ours measured it. The claim arrived from llm-security (coord 20260809T201048Z) and was written down here as a commons finding in three places. Measured against the guard's own tree: severity.mjs has never appeared in src/llm_ingestion_guard/injection_lexicon.json at any point in that file's history (git log -S: no commits), and at 0bf0729 - the commit our manifest pins - the only tree-wide occurrence is docs/PLAN.md:114, correctly about the report module. Their only lexicon source statement is the note at line 3, and it names injection-patterns.mjs. The right file. No detection data moves. hybrid severity is still high, still sourced to injection-patterns.mjs:274-281 at b0de0ca; severity.mjs still has zero occurrences of the four pattern arrays, re-measured today. Only the sentence about the other repository falls. Marked in place, not edited away. The claim survived review because it came bundled with a correct measurement of the same question - a wrong citation to a right value, which is exactly what the field it lived in was written to warn against. lexicon/injection-lexicon.json 0.5.0 -> 0.5.1 (provenance metadata only). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DWKkmKQBsWtrkod4MusUBC
This commit is contained in:
parent
0ffee85a4b
commit
f082a91995
3 changed files with 68 additions and 8 deletions
42
CHANGELOG.md
42
CHANGELOG.md
|
|
@ -9,6 +9,43 @@ Versioning note: the repository tag versions **the contract** (file set, key nam
|
||||||
case ids, disposition semantics). Each JSON file additionally carries its own
|
case ids, disposition semantics). Each JSON file additionally carries its own
|
||||||
`"version"` field, bumped when that file changes.
|
`"version"` field, bumped when that file changes.
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Retracted
|
||||||
|
|
||||||
|
- **The claim that the Python guard's port cites `severity.mjs` for hybrid severity.** It is
|
||||||
|
false. It was carried in three places — `lexicon/injection-lexicon.json`
|
||||||
|
(`families[hybrid].severity_provenance.not_from`), `docs/lexicon-port-divergence.md`
|
||||||
|
(§ *Severity: the 8 hybrid patterns*), and the `[0.1.0]` entry below — and it was never
|
||||||
|
measured here. It restated an assertion received from `llm-security` (coord message
|
||||||
|
`20260809T201048Z`) as a commons finding.
|
||||||
|
|
||||||
|
Measured against the guard's own tree, which `llm-ingestion-pipeline-security` asked for
|
||||||
|
twice before this was checked: `severity.mjs` has **never** appeared in
|
||||||
|
`src/llm_ingestion_guard/injection_lexicon.json` at any point in that file's history
|
||||||
|
(`git log -S` returns no commits), and at `0bf0729` — the commit
|
||||||
|
`conformance/manifest.json` pins — the only tree-wide occurrence is `docs/PLAN.md:114`,
|
||||||
|
correctly attributing the *report* module to `output.mjs` + `severity.mjs`. The guard's
|
||||||
|
only source statement for the lexicon is the `note` at `injection_lexicon.json:3`, and it
|
||||||
|
names `injection-patterns.mjs`.
|
||||||
|
|
||||||
|
**No detection data moves.** `families[hybrid].severity` is still `high`, still sourced to
|
||||||
|
`injection-patterns.mjs:274-281`, re-verified at `b0de0ca`; `severity.mjs` still contains
|
||||||
|
zero occurrences of `CRITICAL_PATTERNS`, `HIGH_PATTERNS`, `MEDIUM_PATTERNS` and
|
||||||
|
`HYBRID_PATTERNS`, re-measured the same day. Only the sentence about the *other* repository
|
||||||
|
falls.
|
||||||
|
|
||||||
|
The retraction is marked in place rather than edited away, and it is worth naming why this
|
||||||
|
one survived review: the claim arrived bundled with a correct measurement of the same
|
||||||
|
question, from a repository that had done its half properly. The correct half carried the
|
||||||
|
incorrect half past the check — which is precisely the defect
|
||||||
|
`severity_provenance.not_from` was written to warn about, one level up.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- `lexicon/injection-lexicon.json` **0.5.0 → 0.5.1** — provenance metadata only; no pattern,
|
||||||
|
id, alias, family or severity value changes.
|
||||||
|
|
||||||
## [0.1.0] — 2026-08-10
|
## [0.1.0] — 2026-08-10
|
||||||
|
|
||||||
Initial extraction. Runtime-neutral detection data, the finding contract, and a conformance
|
Initial extraction. Runtime-neutral detection data, the finding contract, and a conformance
|
||||||
|
|
@ -224,6 +261,11 @@ mutation, not a weak fixture, and it is recorded as such rather than rounded up.
|
||||||
`severity_provenance.not_from` records the miss explicitly: a wrong citation to a right
|
`severity_provenance.not_from` records the miss explicitly: a wrong citation to a right
|
||||||
value is the harder defect to catch later.
|
value is the harder defect to catch later.
|
||||||
|
|
||||||
|
> **Correction 2026-08-10 (see Unreleased):** the two sentences about *the guard's* citation
|
||||||
|
> are false and were never measured here. The guard's port cites `injection-patterns.mjs`,
|
||||||
|
> the right file. Everything above about `severity.mjs` containing no injection-family
|
||||||
|
> severity, and about where the value actually lives, stands and has been re-measured.
|
||||||
|
|
||||||
`pattern_id_space.not_yet_confirmed` is replaced by `ratification`. Both seeding runtimes
|
`pattern_id_space.not_yet_confirmed` is replaced by `ratification`. Both seeding runtimes
|
||||||
agreed on 2026-08-09 — `llm-security` ratified the 0.2.0 proposal as-is and treats an id
|
agreed on 2026-08-09 — `llm-security` ratified the 0.2.0 proposal as-is and treats an id
|
||||||
change as breaking on the same terms, and the guard confirmed the space its own port
|
change as breaking on the same terms, and the guard confirmed the space its own port
|
||||||
|
|
|
||||||
|
|
@ -166,12 +166,29 @@ differences in the table above are closed.
|
||||||
|
|
||||||
The part worth keeping is where the value lives. It is not a field. The engine assigns it by
|
The part worth keeping is where the value lives. It is not a field. The engine assigns it by
|
||||||
pushing `HYBRID_PATTERNS` matches straight into the `high` bucket at
|
pushing `HYBRID_PATTERNS` matches straight into the `high` bucket at
|
||||||
`injection-patterns.mjs:274-281`. The guard's port cites `severity.mjs` — a file that
|
`injection-patterns.mjs:274-281`. `severity.mjs` contains **no injection-family severity at
|
||||||
contains **no injection-family severity at all**. So the guard held the right value behind a
|
all** — re-measured 2026-08-10 at `b0de0ca`: `CRITICAL_PATTERNS`, `HIGH_PATTERNS`,
|
||||||
citation that leads nowhere, and a reviewer following that citation to check the number would
|
`MEDIUM_PATTERNS` and `HYBRID_PATTERNS` appear there zero times.
|
||||||
have found nothing and drawn no conclusion. Refusing to copy it was the right call for a
|
|
||||||
reason better than the one given at the time: not merely that a port is second-hand, but that
|
~~**The guard's port cites `severity.mjs`.** So the guard held the right value behind a
|
||||||
this particular port could not have read what it claimed to.
|
citation that leads nowhere, and refusing to copy it was right for a better reason than the
|
||||||
|
one given at the time: this particular port could not have read what it claimed to.~~
|
||||||
|
**Retracted 2026-08-10. The guard's port cites the right file.** This paragraph was never
|
||||||
|
measured here; it restated an assertion received from `llm-security`
|
||||||
|
(`20260809T201048Z`: *"Guardens port satte riktig verdi, men kunne ikke ha lest den fra fila
|
||||||
|
den oppgir"*) as a commons finding. Measured against the guard's own tree: `severity.mjs`
|
||||||
|
has **never** appeared in `src/llm_ingestion_guard/injection_lexicon.json` at any point in
|
||||||
|
that file's history (`git log -S` returns no commits), and at `0bf0729` — the commit
|
||||||
|
`conformance/manifest.json` pins — the only tree-wide occurrence is `docs/PLAN.md:114`,
|
||||||
|
correctly attributing the *report* module to `output.mjs` + `severity.mjs`. The guard's only
|
||||||
|
source statement for the lexicon is the `note` at `injection_lexicon.json:3`, and it names
|
||||||
|
`injection-patterns.mjs`.
|
||||||
|
|
||||||
|
Refusing to copy the value was still the right call — but for the plain reason, that a port
|
||||||
|
is second-hand evidence, not for the sharper one claimed above. The sharper reason was itself
|
||||||
|
a wrong citation to a right value, which is the defect this section was written to warn
|
||||||
|
about. It survived here because it arrived from a repository that had measured the *other*
|
||||||
|
half of the claim correctly, and the correct half carried the incorrect half past review.
|
||||||
|
|
||||||
## What this does not show
|
## What this does not show
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
{
|
{
|
||||||
"version": "0.5.0",
|
"version": "0.5.1",
|
||||||
"id": "injection-lexicon",
|
"id": "injection-lexicon",
|
||||||
"description": "Prompt-injection pattern lexicon: the four pattern families a detector matches against untrusted text, and the severity family each belongs to. Data only. The variant building that feeds these patterns (normalisation, homoglyph folding, rot13, unicode-tag escalation) and the buried-payload heuristic are engine behaviour and are deliberately NOT described here.",
|
"description": "Prompt-injection pattern lexicon: the four pattern families a detector matches against untrusted text, and the severity family each belongs to. Data only. The variant building that feeds these patterns (normalisation, homoglyph folding, rot13, unicode-tag escalation) and the buried-payload heuristic are engine behaviour and are deliberately NOT described here.",
|
||||||
"owasp": "LLM01",
|
"owasp": "LLM01",
|
||||||
|
|
@ -894,7 +894,8 @@
|
||||||
"source_lines": "274-281",
|
"source_lines": "274-281",
|
||||||
"source_commit": "b0de0ca6d86ce697f39669d177c2c2654c280128",
|
"source_commit": "b0de0ca6d86ce697f39669d177c2c2654c280128",
|
||||||
"verified": "directly, by reading the module at the pinned commit",
|
"verified": "directly, by reading the module at the pinned commit",
|
||||||
"not_from": "scanners/lib/severity.mjs, which was the first file asked about and which contains no injection-family severity at all - grep for CRITICAL_PATTERNS, HIGH_PATTERNS, MEDIUM_PATTERNS or HYBRID_PATTERNS there returns nothing. The Python guard's port carries the correct value but cites that file, so a consumer following its citation would find no answer. Recorded because a wrong citation to a right value is the harder defect to notice."
|
"not_from": "scanners/lib/severity.mjs, which was the first file asked about and which contains no injection-family severity at all - grep for CRITICAL_PATTERNS, HIGH_PATTERNS, MEDIUM_PATTERNS or HYBRID_PATTERNS there returns nothing at the pinned commit. That part stands and was re-measured 2026-08-10.",
|
||||||
|
"retracted_2026-08-10": "This field also said: 'The Python guard's port carries the correct value but cites that file, so a consumer following its citation would find no answer.' RETRACTED - it is false, and it was never measured here. It restated an assertion received from llm-security (coord message 20260809T201048Z) as if it were a commons finding. Measured against the guard's own tree: severity.mjs has never appeared in src/llm_ingestion_guard/injection_lexicon.json at any point in that file's history (git log -S returns no commits), and at 0bf0729 - the commit conformance/manifest.json pins - the only tree-wide occurrence is docs/PLAN.md:114, which correctly attributes the report module to output.mjs + severity.mjs and says nothing about injection-family severity. The guard's only source statement for the lexicon is the note field at injection_lexicon.json:3, and it cites injection-patterns.mjs - the right file. Kept rather than deleted because this repository's stated reason for recording the original claim was that a wrong citation to a right value is the harder defect to notice, and that turned out to describe this record itself."
|
||||||
},
|
},
|
||||||
"count": 8,
|
"count": 8,
|
||||||
"patterns": [
|
"patterns": [
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue