Kjell Tore Guttormsen
f68f1e19d5
Advisor review found six gaps the session's own checks did not cover. The one
that mattered was outward-facing: the divergence report told the guard repo
that two sub-agent bounds were undocumented. That was wrong. lexicon.py's
module docstring documents them explicitly under "Bounded token gaps", and
git log -S dates {0,12} to the original port commit f397cd9 and [^><] to the
ReDoS fix cff0437. Every divergence on the guard's side is documented and
traceable. The claim rested on two sed slices of one file; an absence claim
needs a search over the whole repository. Corrected here and by coord.
lexicon/injection-lexicon.json 0.2.0 -> 0.3.0:
- pattern_id_space.alias_evidence records the two aliases separately instead
of averaging them. llm_ingestion_guard is verified — coverage.py asserts on
that exact string, so it is demonstrably what a guard finding carries.
llm_security is not: it is the pattern table's name, the finding producer
was never supplied, and the known Node finding shape uses title, not label.
- normalisations[].affects now keys on id, with the prose names kept beside it
as affects_labels. An internal cross-reference on label was a second
identity space inside the file the id exists to unify.
Detection data unmoved again: labels, patterns, flags and the ids and aliases
added in 7b70f5b are all byte-identical in sequence; 166/166 Node compiles.
Also: the README lexicon row described thematic families the file does not
have (they are severity families; the theme is the id prefix), and the
conformance convertibility table gained its missing second condition — a case
is buildable only if the label it asserts maps to data this repository
publishes. Eleven cases fail that test (entropy, decoded, the sanitize rows,
the OKF scans), so the buildable set is ~94, not 105.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
174 lines
11 KiB
Markdown
174 lines
11 KiB
Markdown
# Lexicon port divergence — commons vs the Python guard
|
||
|
||
**Status: informative.** Nothing here is normative and nothing here changes a data file. It
|
||
records a measured disagreement between two ports of one source table, so the decision can be
|
||
taken where each table is tested. Under this repository's behaviour-preservation invariant,
|
||
a divergence found here is **reported, not fixed**.
|
||
|
||
Produced 2026-08-09. Every number below came from a command; the scripts live in the session
|
||
scratchpad rather than in this repository, because executable code here would breach the
|
||
charter. They are reproducible from the method column.
|
||
|
||
## What was compared
|
||
|
||
| Side | Artefact | Version |
|
||
| --- | --- | --- |
|
||
| commons | [`lexicon/injection-lexicon.json`](../lexicon/injection-lexicon.json) | file `version` as committed |
|
||
| guard | `llm-ingestion-pipeline-security` `src/llm_ingestion_guard/injection_lexicon.json` | lexicon `version` 1.0, repo v0.3.4, commit `0bf0729` |
|
||
|
||
Both are **ports of the same file**: `llm-security/scanners/lib/injection-patterns.mjs`. The
|
||
guard's JSON says so in its own `note` field — *"Injection lexicon ported from llm-security
|
||
injection-patterns.mjs. Single source of truth."* Commons extracted the same table from an
|
||
operator dump of that module.
|
||
|
||
That is what makes the comparison worth running. These are not two different detectors that
|
||
happen to overlap; they are two transcriptions of one table, and where they disagree, they
|
||
disagree about what the same source says.
|
||
|
||
## Result
|
||
|
||
| Measure | Method | Result |
|
||
| --- | --- | --- |
|
||
| Pattern count, both sides | count entries | 83 and 83 |
|
||
| Label correspondence | match commons `label` to guard `desc`, em-dash normalised to hyphen | **83/83** |
|
||
| Regex source byte-identical | string compare | **64/83** |
|
||
| Differing regex text | string compare | 19 |
|
||
| — of those, provably equivalent | unescape commons' JS-isms (`\/` → `/`, `\uXXXX` → literal) and compare for string identity | **6/6 identical** |
|
||
| — of those, behaviourally divergent | differential match-set comparison (offsets + matched text), targeted corpus per pattern family | **13**, each with a concrete witness input |
|
||
| Total input comparisons | count | 401 |
|
||
| Flags | compare declared flags | **0 differences** |
|
||
| Severity / family | commons family vs guard `severity` | **8 differences** (all `hybrid`) |
|
||
|
||
64 identical + 6 escaping-only + 13 divergent = 83.
|
||
|
||
The 6 escaping-only pairs are equivalent by **string identity after unescaping**, not merely
|
||
by absence of a counter-example: `<\/system>` vs `</system>`, `text\/html` vs `text/html`,
|
||
`(?:https?:\/\/…)` vs `(?:https?://…)`, `(?:~\/)?` vs `(?:~/)?`, and two `\uXXXX`-vs-literal
|
||
character classes. Commons writes code points as escapes by convention; the guard's `note`
|
||
records the same choice for non-ASCII.
|
||
|
||
## The 13, and which side misses what
|
||
|
||
Divergence is **not one-directional**. Each side has inputs the other's pattern does not match.
|
||
|
||
| Pattern | Direction | Witness input |
|
||
| --- | --- | --- |
|
||
| `html-obfuscation: display:none element with content` | commons misses; span differs | attribute run padded past 256 chars |
|
||
| `html-obfuscation: visibility:hidden element` | span only | `<div data-x="<" style="visibility:hidden">x</div>` |
|
||
| `html-obfuscation: off-screen positioned element` | span only | same shape, `position:absolute` |
|
||
| `html-obfuscation: zero font-size element` | span only | same shape, `font-size:0` |
|
||
| `html-obfuscation: zero opacity element` | span only | same shape, `opacity:0` |
|
||
| `html-obfuscation: zero-size overflow-hidden element` | span only | same shape, `height:0;overflow:hidden` |
|
||
| `html-obfuscation: injection in aria-label attribute` | **commons misses** | `<a aria-label="` + 300 × `a` + `ignore">x</a>` |
|
||
| `markdown: suspicious instruction in link-reference comment` | **guard misses** | `[//]: # (a (b) ignore)` |
|
||
| `markdown: injection payload in link anchor text` | **guard misses** | `[click ignore this](http://e.com/a(b))` |
|
||
| `sub-agent: spawn instruction with dangerous capability keywords` | **guard misses** | `create an agent that` + 20 filler words + `execute` |
|
||
| `sub-agent: delegation with safety bypass instruction` | **guard misses** | `delegate to a new agent` + 200 chars + `bypass` |
|
||
| `hybrid-xss: <script> tag in content (agent context XSS)` | **commons misses**; span differs | `<script>alert(1)` (unclosed), `<script src=x.js>` |
|
||
| `hybrid-xss: iframe with executable src (agent context XSS)` | **guard misses** | `<iframe data-x="<" src="javascript:alert(1)">` |
|
||
|
||
"Span only" means both sides produce a match on the same input but over different extents —
|
||
the guard's match starts at an interior `<`. Whether that matters depends on whether a
|
||
consumer reports offsets or evidence text; it does not change whether a finding is raised.
|
||
|
||
The commons-side misses were confirmed in a real JS engine (Node v25.8.2, `RegExp` built from
|
||
the committed JSON), not only in the Python harness used for the differential.
|
||
|
||
## Why they diverge: two different ReDoS mitigations of one table
|
||
|
||
This is not drift, and framing it as a bug in either repository would be wrong.
|
||
|
||
Both ports have been hardened against catastrophic backtracking, by **different strategies**:
|
||
|
||
- **The Node side bounds the run.** `[^"]{0,256}`, `[^>]{1,256}`. Cost: an attacker who pads
|
||
the attribute past 256 characters falls out of the pattern.
|
||
- **The guard excludes the anchor character.** `[^><]`, `[^\]\[]`, `[^)(]`. Cost: content that
|
||
legitimately contains that character stops matching.
|
||
|
||
**Every divergence on the guard's side is documented at source, and traceable to the commit
|
||
that introduced it.** An earlier draft of this file claimed the two sub-agent bounds were not;
|
||
that was wrong, and it was wrong because the search behind it never looked outside the
|
||
CHANGELOG. Both mechanisms are named in `lexicon.py`'s own module docstring:
|
||
|
||
> **Bounded token gaps** — the two sub-agent patterns whose seed form nested `.*?` are ported
|
||
> with `(?:\S+\s+){0,N}?`.
|
||
>
|
||
> **Anchor exclusion** — […] Measured across all 83 patterns arm by arm, two markdown patterns
|
||
> had this defect; both now exclude the anchor character from the run.
|
||
|
||
`git log -S` separates the two: the eight `[^><]` patterns (six html-obfuscation, two
|
||
hybrid-xss) arrived with `cff0437`, *"fix(output): 19 quadratic regex runs on the output path"*
|
||
— the v0.3.2 campaign, whose CHANGELOG describes exactly this remedy (*"exclude the character
|
||
that opens the pattern's own anchor (`[` for markdown, `<` for tags)"*) across a sweep of
|
||
*"150 patterns across 11 tables"*. The `{0,12}` / `{0,120}` sub-agent bounds are older still:
|
||
they arrived with the original port commit `f397cd9`, so they were never a divergence
|
||
introduced later — they are how that table was transcribed in the first place.
|
||
|
||
The v0.3.4 entry also states the measured recall cost, naming precisely the two exceptions this
|
||
comparison rediscovered: *"URLs containing a literal `(` inside a markdown link target and
|
||
comment bodies containing a literal `(` before the keyword."*
|
||
|
||
Worth recording, because it anticipates the criticism the Node side invites: the guard
|
||
considered bounding those runs and **rejected it**, on the grounds that *"the content is
|
||
attacker-controlled, so padding past a bound would be a one-line bypass."* That is the same
|
||
objection the `{0,256}` witness above demonstrates against the Node table. The two projects
|
||
reached opposite conclusions from the same reasoning, which is the substance of the
|
||
disagreement — not an oversight on either side.
|
||
|
||
Neither strategy is free, and neither is obviously right. That is the decision the two owning
|
||
repositories have to take, and it is not commons' to take for them.
|
||
|
||
## Severity: the 8 hybrid patterns
|
||
|
||
Commons records the `hybrid` family with `severity: null` and a note that the seed dump did
|
||
not supply it, so a consumer **MUST NOT** assume one. The guard's port assigns all eight
|
||
`high`.
|
||
|
||
**This has deliberately not been copied into commons.** The guard's port is a second-hand
|
||
transcription, not the producing module; adopting its value would convert a documented gap
|
||
into an unverified claim, which is the defect class this repository's changelog already
|
||
records twice. It is reported instead, as a strong hint that the Node source assigns `high`,
|
||
for `llm-security` to confirm from the module.
|
||
|
||
## What this does not show
|
||
|
||
- **Not that either runtime misses an attack.** All witness payloads were re-tested against
|
||
every pattern table commons holds (111 compiled rules: 83 lexicon, 28 from
|
||
`active-content.json` and `secret-egress.json`). All of them still produce a finding, via
|
||
`active-content: constructs.raw-html`. The divergence is in the **finding set** — which
|
||
labels are raised — not in whether anything is raised at all. That still matters, because a
|
||
finding set is exactly what a `conformance/expected.json` would encode.
|
||
- **Not that `llm-security` behaves as described here.** Only its extracted pattern table was
|
||
available. Whether the Node engine runs an active-content table that compensates cannot be
|
||
determined from this repository.
|
||
- **Not dump-to-module fidelity.** Every check above proves the two *ports* agree or disagree.
|
||
That either matches `injection-patterns.mjs` remains `llm-security`'s assertion,
|
||
reproducible only in a session with read access to it.
|
||
- **Not exhaustive.** The corpus is targeted per pattern family, 401 comparisons. Absence of a
|
||
witness proves nothing except for the 6 escaping-only pairs, which are settled by string
|
||
identity rather than by the corpus.
|
||
|
||
## Consequence for `conformance/`
|
||
|
||
An `expected.json` names findings. Naming a finding needs a stable id, and the two runtimes do
|
||
not have one: the same pattern is `override:ignore-previous` in the guard and
|
||
`override: ignore previous instructions` in the Node table. The guard's JSON happens to carry
|
||
both — `id` and `desc` — which is evidence that a commons-owned id is achievable rather than
|
||
speculative.
|
||
|
||
So the id question is a **prerequisite** for the corpus, not a parallel task: until commons
|
||
owns a pattern id both ports map to, no fixture can be written, including for the 64 patterns
|
||
that are byte-identical. And for the 13 divergent patterns a fixture cannot be authored at all
|
||
without first deciding whose recall cost is the contract.
|
||
|
||
**Resolved for the first half, 2026-08-09 (operator decision).** `lexicon/injection-lexicon.json`
|
||
0.2.0 now carries a commons-owned `id` per pattern, plus an `aliases` object naming what each
|
||
seeding runtime calls it. The id was **adopted verbatim from the guard's port**, not invented
|
||
here — that port already carried both names, so the mapping came from source data. The
|
||
detection data is provably unmoved: stripping `id`, `aliases` and the new `pattern_id_space`
|
||
block reproduces the previous committed file **byte for byte** (23 566 bytes, identical).
|
||
|
||
Neither runtime has agreed to it yet — both were asked by coord the same day — so the file
|
||
records it as this repository's proposal rather than a ratified contract. The second half of
|
||
the blocker stands: the 13 divergent patterns still have no agreed expected behaviour, so
|
||
their fixtures cannot be authored even now that they can be named.
|