open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

docs(lexicon): split the alias evidence, correct a false absence claim

Browse source 
Advisor review found six gaps the session's own checks did not cover. The one
that mattered was outward-facing: the divergence report told the guard repo
that two sub-agent bounds were undocumented. That was wrong. lexicon.py's
module docstring documents them explicitly under "Bounded token gaps", and
git log -S dates {0,12} to the original port commit f397cd9 and [^><] to the
ReDoS fix cff0437. Every divergence on the guard's side is documented and
traceable. The claim rested on two sed slices of one file; an absence claim
needs a search over the whole repository. Corrected here and by coord.

lexicon/injection-lexicon.json 0.2.0 -> 0.3.0:

- pattern_id_space.alias_evidence records the two aliases separately instead
  of averaging them. llm_ingestion_guard is verified — coverage.py asserts on
  that exact string, so it is demonstrably what a guard finding carries.
  llm_security is not: it is the pattern table's name, the finding producer
  was never supplied, and the known Node finding shape uses title, not label.
- normalisations[].affects now keys on id, with the prose names kept beside it
  as affects_labels. An internal cross-reference on label was a second
  identity space inside the file the id exists to unify.

Detection data unmoved again: labels, patterns, flags and the ids and aliases
added in 7b70f5b are all byte-identical in sequence; 166/166 Node compiles.

Also: the README lexicon row described thematic families the file does not
have (they are severity families; the theme is the id prefix), and the
conformance convertibility table gained its missing second condition — a case
is buildable only if the label it asserts maps to data this repository
publishes. Eleven cases fail that test (entropy, decoded, the sanitize rows,
the OKF scans), so the buildable set is ~94, not 105.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 21:56:45 +02:00
commit f68f1e19d5
5 changed files with 78 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

21
docs/extraction-plan.md
View file

@ -279,8 +279,25 @@ engine composes two findings a particular way. This repository publishes data an
explicitly does not own an API, so those rows belong to the guard's own suite, the same way
the `Planned` rows in the README are named rather than faked.
Of the 103 `_scan_case` rows, 83 are one-per-lexicon-pattern. Those are also the rows most
affected by the id question below.
**Convertibility has a second condition, and it is the one easy to miss.** A case is a
candidate only if the probe is text-in / findings-out **and** the label it asserts maps to data
this repository actually publishes. A fixture asserting a label commons has no table for would
quietly extend the contract to cover engine behaviour the charter excludes — the corpus would
then be testing something this repository does not define.
| Group | Cases | Asserted label maps to commons data? |
| --- | --- | --- |
| `lexicon` | 83 | Yes — `lexicon/injection-lexicon.json`, now via the `id` field |
| `active` | 7 | Yes — `signatures/active-content.json` |
| `carrier` | 8 | Partly. `output:*-present` and `lexicon:unicode-tags-present` rest on the code-point tables in `codepoints/carriers.json`. The five `sanitize:*` rows assert **stripper** behaviour, which is engine. |
| `egress` | 1 | Yes — `signatures/secret-egress.json` |
| `entropy` | 2 | **No.** Shannon-entropy detection is the guard's `entropy.py`; commons publishes no entropy table. (`active-content.json` carries an opacity floor — a different thing, for a different pass.) |
| `decoded` | 2 | **No.** These assert a finding *after* a decode pass. The decode order is exactly what `spec/decode-pipeline.md` would fix, and it is Planned because it is engine. |
| `okf` (`_scan_case_okf`) | 2 | No — the OKF document model is not published here. |
So the buildable set is smaller than 105: roughly 83 + 7 + 3 + 1 ≈ 94, minus the 13 lexicon
patterns whose expected behaviour is still disputed. The rest are named here so a later session
does not rediscover them as a gap.
**Blocking issue found while counting.** The guard and the Node table are two ports of one
source file and they do not agree: 64 of 83 patterns byte-identical, 6 differing only by

33
docs/lexicon-port-divergence.md
View file

@ -85,12 +85,35 @@ Both ports have been hardened against catastrophic backtracking, by **different
- **The guard excludes the anchor character.** `[^><]`, `[^\]\[]`, `[^)(]`. Cost: content that
legitimately contains that character stops matching.
The guard's campaign is documented in its own CHANGELOG (v0.3.2 and v0.3.4: *"exclude the
character that opens the pattern's own anchor"*, a sweep over *"150 patterns across 11
tables"*), including its measured recall cost — it names exactly the two exceptions this
**Every divergence on the guard's side is documented at source, and traceable to the commit
that introduced it.** An earlier draft of this file claimed the two sub-agent bounds were not;
that was wrong, and it was wrong because the search behind it never looked outside the
CHANGELOG. Both mechanisms are named in `lexicon.py`'s own module docstring:
> **Bounded token gaps** — the two sub-agent patterns whose seed form nested `.*?` are ported
> with `(?:\S+\s+){0,N}?`.
>
> **Anchor exclusion** — […] Measured across all 83 patterns arm by arm, two markdown patterns
> had this defect; both now exclude the anchor character from the run.
`git log -S` separates the two: the eight `[^><]` patterns (six html-obfuscation, two
hybrid-xss) arrived with `cff0437`, *"fix(output): 19 quadratic regex runs on the output path"*
— the v0.3.2 campaign, whose CHANGELOG describes exactly this remedy (*"exclude the character
that opens the pattern's own anchor (`[` for markdown, `<` for tags)"*) across a sweep of
*"150 patterns across 11 tables"*. The `{0,12}` / `{0,120}` sub-agent bounds are older still:
they arrived with the original port commit `f397cd9`, so they were never a divergence
introduced later — they are how that table was transcribed in the first place.
The v0.3.4 entry also states the measured recall cost, naming precisely the two exceptions this
comparison rediscovered: *"URLs containing a literal `(` inside a markdown link target and
comment bodies containing a literal `(` before the keyword."* The bounded `{0,12}` / `{0,120}`
sub-agent quantifiers produce misses that the guard's exception list does not mention.
comment bodies containing a literal `(` before the keyword."*
Worth recording, because it anticipates the criticism the Node side invites: the guard
considered bounding those runs and **rejected it**, on the grounds that *"the content is
attacker-controlled, so padding past a bound would be a one-line bypass."* That is the same
objection the `{0,256}` witness above demonstrates against the Node table. The two projects
reached opposite conclusions from the same reasoning, which is the substance of the
disagreement — not an oversight on either side.
Neither strategy is free, and neither is obviously right. That is the decision the two owning
repositories have to take, and it is not commons' to take for them.