open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

fix(lexicon): retract two false provenance claims, reach 83/83 to source

Browse source 
The lexicon was seeded from a transcribed dump. With the source module now
readable read-only at b0de0ca on the public remote, it was imported in Node
and compared entry by entry - RegExp source, flags, label, family, array
order. That check contradicted the file twice, in mirror-image ways.

The file claimed it had rewritten raw code points into \uXXXX escapes in
homoglyph:cyrillic-latin-mix and unicode:zero-width-in-word, and carried
byte_identical_to_source: false for both. The module already writes them
escaped. Nothing was ever rewritten. The stored pattern text was correct
throughout - only the account of its origin was wrong, because the dump had
rendered the module's escapes as the characters they denote and this
repository escaped them back.

The same inversion ran the other way in multi-lang:french, which stored a raw
accented Latin e where the module writes the é escape. That was the one
pattern of 83 not byte-identical to source, and it is corrected here, making
it 83/83. Both spellings are the same regular expression: compiled in Node
bare and under u, and in Python re, over accented, unaccented, uppercase and
non-matching French input, with identical match offsets. No behaviour moved.

Exactly one pattern entry differs from HEAD, and no pattern in the file
contains a non-ASCII byte any more - which matches the module, whose regex
literals are ASCII throughout. A third claim fell with them: the translation
note asserting that the leetspeak and multi-language patterns embed accented
Latin directly was wrong in both halves.

normalisations is now [] with a normalisations_note, the convention already
in signatures/secret-egress.json. A new source_fidelity block carries the
counts, the method, the verified class membership, and both retractions in
full - retracted, not deleted. The equivalence evidence behind the withdrawn
claim (692 Node comparisons, 236 Python) is still a true statement about raw
versus escaped; it is just no longer load-bearing.

Provenance upgraded accordingly: source_commit pinned, and the caveat saying
these checks only proved agreement with a dump is gone, because they no
longer do.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SNMcqrfNyoLRQ7qXUFZnb9
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 22:39:32 +02:00
commit 344bc52adf
2 changed files with 75 additions and 41 deletions
Download patch file Download diff file Expand all files Collapse all files

29
CHANGELOG.md
View file

@ -53,6 +53,35 @@ checks that were not run instead of attaching a caveat to a pass.
### Changed
- `lexicon/injection-lexicon.json` **0.3.0 → 0.4.0** — verified against the source module
instead of against the dump it was transcribed from, and **two false provenance claims
retracted**. The source is now pinned: `b0de0ca` on the public remote, imported in Node
and compared entry by entry on `source`, `flags` and `label`.
The result is **83/83 byte-identical to source**, which is not what the file previously
claimed. It said two patterns had been rewritten from raw code points into `\uXXXX`
escapes; the module already writes them escaped, so nothing had been rewritten. The stored
pattern text was right the whole time — only the account of where it came from was wrong.
The dump had rendered the module's escapes as the characters they denote, and this
repository re-escaped them, arriving at the correct bytes by way of an incorrect story.
The same inversion ran the other way in `multi-lang:french`, which carried the class
spelled with a raw accented Latin `e` where the module writes it as the escape
`\u00e9` inside the same character class.
That was the one pattern of 83 not byte-identical to source,
and it is corrected. The two spellings are the same regular expression — verified in Node
bare and under `u`, and in Python `re`, over accented, unaccented, uppercase and
non-matching French input, with identical match offsets — so **no behaviour moved**. No
pattern in the file contains a non-ASCII byte now, matching the module, whose regex
literals are pure ASCII throughout.
Structurally: `normalisations` is now `[]` with a `normalisations_note`, matching the
convention already used in `signatures/secret-egress.json`, and a new `source_fidelity`
block carries the counts, the method, the verified class membership, and both retractions
in full. Retracted claims are recorded rather than deleted — the earlier equivalence
evidence (692 Node comparisons, 236 Python) remains true, it is simply no longer
load-bearing.
- `lexicon/injection-lexicon.json` **0.2.0 → 0.3.0** — the two aliases are no longer presented
as equally backed. `pattern_id_space.alias_evidence` now records each one separately:
`llm_ingestion_guard` is **verified** (the guard's coverage matrix asserts on that exact

87
lexicon/injection-lexicon.json
View file

@ -1,5 +1,5 @@
{
"version": "0.3.0",
"version": "0.4.0",
"id": "injection-lexicon",
"description": "Prompt-injection pattern lexicon: the four pattern families a detector matches against untrusted text, and the severity family each belongs to. Data only. The variant building that feeds these patterns (normalisation, homoglyph folding, rot13, unicode-tag escalation) and the buried-payload heuristic are engine behaviour and are deliberately NOT described here.",
"owasp": "LLM01",
@ -23,7 +23,7 @@
}
}
},
"$comment": "Extracted without behaviour change from llm-security/scanners/lib/injection-patterns.mjs, delivered as operator dump 2/2 through the local coord mailbox on 2026-08-09. Only the four pattern arrays are data. The dump states explicitly that checkCognitiveLoadTrap (the buried-after-2000-chars logic) and scanForInjection's variant building are ENGINE logic that never moves here; they are absent by intent, not by omission. Array order is preserved exactly as delivered.",
"$comment": "Extracted without behaviour change from llm-security/scanners/lib/injection-patterns.mjs. Seeded 2026-08-09 from an operator dump delivered through the local coord mailbox; re-verified the same day against the module file itself at commit b0de0ca, which corrected two transcription artefacts the dump had introduced (see source_fidelity.retracted). Only the four pattern arrays are data. checkCognitiveLoadTrap (the buried-after-2000-chars logic) and scanForInjection's variant building are ENGINE logic that never moves here; they are absent by intent, not by omission. Array order is preserved exactly as in the module.",
"provenance": {
"source_repo": "llm-security",
"source_files": [
@ -35,11 +35,11 @@
"MEDIUM_PATTERNS",
"HYBRID_PATTERNS"
],
"source_delivery": "operator dump 2/2, coord message from llm-security, 2026-08-09",
"source_commit": "unknown - not supplied with the dump",
"verified": "differentially, against the dump",
"source_delivery": "operator dump 2/2, coord message from llm-security, 2026-08-09; superseded by a direct read of the module at the pinned commit below",
"source_commit": "b0de0ca6d86ce697f39669d177c2c2654c280128",
"source_remote": "ssh://git@git.fromaitochitta.com/open/llm-security.git",
"verified": "differentially, against the module file at b0de0ca - all 83 patterns compared on source, flags and label, in family and array order, 2026-08-09. See source_fidelity.",
"evidence_limits": [
"The dump is a transcription of the source module, not the module file itself. Every check recorded for this file therefore proves that this JSON agrees with the DUMP. That the dump agrees with the module is llm-security's assertion, and is not a result reproduced here.",
"The severity the engine assigns to the hybrid family was not supplied. See families[].severity_note.",
"Whether a consumer applies these patterns to raw or to normalised text is engine behaviour and is not fixed by this file. Two runtimes matching identical patterns against differently normalised text will still disagree on identical input."
]
@ -62,45 +62,50 @@
"Python (`re`): compile with re.I where flags contain `i` and re.M where they contain `m`. No rewriting is needed; verified by compiling all 83.",
"Nine patterns contain `\\/` - a redundant escape that a JavaScript regex LITERAL requires and that `RegExp.prototype.source` preserves. It is kept rather than normalised away because both seed runtimes accept it (Node bare, Node under `u`, and Python `re`), so removing it would break byte-identity to buy nothing. Engines that reject unknown escapes (Go `regexp`, RE2) MUST report these patterns as unsupported rather than skip them silently - a skipped pattern is an invisible false negative.",
"`\\w` and `\\b` are ASCII-only in ECMAScript without `u` but Unicode-aware in Python on `str`. Input that mixes non-ASCII word characters with these patterns can therefore match differently between runtimes. This is a real portability seam. It is recorded rather than normalised, because normalising it would change behaviour in the seed runtime.",
"The leetspeak and multi-language patterns embed non-ASCII literals (accented Latin) directly. Those are visible characters and are left exactly as written."
"No pattern in this file contains a non-ASCII byte. The source module's regex literals are pure ASCII throughout - invisible code points, Cyrillic look-alikes and accented Latin are all written as \\uXXXX in the literal, and this file reproduces that spelling. An earlier version of this note claimed the leetspeak and multi-language patterns embed accented Latin directly; that was wrong in both halves (the leetspeak patterns are ASCII-only, and the one accented pattern is escaped at source). See source_fidelity.retracted."
]
},
"normalisations": [
{
"id": "unicode-escape-invisible-and-confusable",
"affects": [
"homoglyph:cyrillic-latin-mix",
"unicode:zero-width-in-word"
"normalisations": [],
"normalisations_note": "Empty by result, not by omission: all 83 patterns are byte-identical to the source module, so there is no transformation to declare. Through version 0.3.0 this array held one entry; it was retracted as false in 0.4.0. See source_fidelity.",
"source_fidelity": {
"$comment": "This file applies NO transformation to the pattern text. All 83 patterns are byte-identical to the RegExp literal source in llm-security/scanners/lib/injection-patterns.mjs at commit b0de0ca. The `normalisations` array is empty because there is nothing to declare, not because the question was skipped.",
"patterns_total": 83,
"patterns_byte_identical_to_source": 83,
"method": "The module was imported in Node from a read-only clone of the public remote at b0de0ca, and every entry compared in array order on three axes: RegExp.prototype.source, RegExp.prototype.flags, and label. Family membership, array order and per-family counts were compared at the same time. Nothing was read by eye.",
"class_membership_verified": {
"zero_width_class": [
"U+200B",
"U+200C",
"U+200D",
"U+FEFF"
],
"affects_labels": [
"homoglyph: Cyrillic-Latin mixing in adjacent characters",
"unicode: zero-width character inside word (keyword splitting)"
"cyrillic_class": [
"U+0430",
"U+0435",
"U+043E",
"U+0440",
"U+0441",
"U+0456",
"U+0443"
],
"from": "literal U+200B U+200C U+200D U+FEFF, and literal Cyrillic a/e/o/r/s/i/u, inside character classes",
"to": "the same code points written as \\uXXXX escapes",
"reason": "The source literals carry these code points RAW. Four of them are invisible and seven are Latin look-alikes, so in that form the pattern text cannot be reviewed, diffed or copied safely: a reviewer cannot see whether a class holds four code points or five. This repository's convention forbids raw invisible code points outside conformance inputs for exactly this reason. The escaped spelling is the same regular expression to both engines.",
"equivalence_evidence": "Differential match-set comparison (match offsets and matched text) between the raw and the escaped form: Node over 173 adversarial inputs in both bare and `u` mode (692 comparisons), and Python `re` over 118 inputs (236 comparisons). Zero differences. The corpus covers every member of each class, the near-misses deliberately excluded from it (U+00AD, U+2060, U+180E, Cyrillic x, the Cyrillic uppercase set, Greek look-alikes), run boundaries, repeats, and empty input.",
"class_membership_verified": {
"zero_width_class": [
"U+200B",
"U+200C",
"U+200D",
"U+FEFF"
],
"cyrillic_class": [
"U+0430",
"U+0435",
"U+043E",
"U+0440",
"U+0441",
"U+0456",
"U+0443"
],
"$comment": "Enumerated mechanically from the dump bytes, not by eye. Note that the zero-width class here does NOT include U+00AD even though codepoints/carriers.json lists U+00AD as a zero-width carrier, and that the Cyrillic class here is a 7-member set distinct from the 13-member CYRILLIC_CONFUSABLES list in that same file. The three sets are deliberately different in the seed implementation and are NOT reconciled here."
"$comment": "The classes inside `unicode:zero-width-in-word` and `homoglyph:cyrillic-latin-mix`, enumerated mechanically from the module bytes. Note that the zero-width class does NOT include U+00AD even though codepoints/carriers.json lists U+00AD as a zero-width carrier, and that the Cyrillic class is a 7-member set distinct from the 13-member CYRILLIC_CONFUSABLES list in that same file. The three sets are deliberately different in the seed implementation and are NOT reconciled here."
},
"retracted": [
{
"id": "unicode-escape-invisible-and-confusable",
"was": "A `normalisations` entry, present through version 0.3.0, asserting that this file rewrote raw code points as \\uXXXX escapes in `homoglyph:cyrillic-latin-mix` and `unicode:zero-width-in-word`, and carrying byte_identical_to_source: false.",
"why_retracted": "False against the module. The source literals already write those code points as \\uXXXX; nothing was ever rewritten, and both patterns are byte-identical to source. The stored pattern text was correct all along - only the account of where it came from was wrong.",
"cause": "The seed dump rendered the module's escape sequences as the literal code points they denote. Reading that as the source spelling, this repository re-escaped them and arrived at the correct bytes by way of an incorrect story. This is transcription drift, and it is why the file is now verified against the module rather than against a dump.",
"evidence_status": "The differential result recorded for the retracted claim - Node over 173 adversarial inputs in bare and `u` mode (692 comparisons) and Python `re` over 118 inputs (236 comparisons), zero differences - stands as a true statement about raw-versus-escaped equivalence. It is simply no longer load-bearing here, because no such transformation is applied."
},
"byte_identical_to_source": false
}
],
{
"id": "french-escape-inverted",
"was": "`multi-lang:french` carried the character class spelled with raw accented Latin: pr[eé]c[eé]dentes?.",
"why_retracted": "The same transcription inversion in the opposite direction. The module writes pr[e\\u00e9]c[e\\u00e9]dentes? with escapes; this was the only one of the 83 patterns not byte-identical to source. Corrected in 0.4.0 to the module's spelling, which makes the count 83/83.",
"behaviour_impact": "None. The two spellings are the same regular expression. Verified by compiling both forms and comparing match presence and offsets in Node (bare and `u` mode) and in Python `re` over accented, unaccented, uppercase and non-matching French inputs - identical results throughout."
}
]
},
"families": [
{
"id": "critical",
@ -742,7 +747,7 @@
{
"id": "multi-lang:french",
"label": "multi-lang: French \"ignore previous instructions\"",
"pattern": "ignor(?:ez?|er?)\\s+(?:les?\\s+)?instructions?\\s+pr[eé]c[eé]dentes?",
"pattern": "ignor(?:ez?|er?)\\s+(?:les?\\s+)?instructions?\\s+pr[e\\u00e9]c[e\\u00e9]dentes?",
"flags": "i",
"aliases": {
"llm_security": "multi-lang: French \"ignore previous instructions\"",