open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

docs: measure the lexicon port divergence, correct the case count

Browse source 
The guard's coverage.py turned out to be readable from a session here — the
read boundary covers llm-security only. Counting it instead of estimating
corrected two published numbers and surfaced the real conformance blocker.

Counted, not estimated: CORE_CASES holds 134 cases (128 caught, 6 gap), not
the "126 classes + 4 gaps-must-hold" the extraction plan claimed. 105 are
convertible to static input/expected; 29 assert a runtime API surface this
repository does not own, and are named as out of scope rather than faked.

Measured while counting, and the reason the corpus is still blocked: the
commons lexicon and the guard's are two ports of one source file, and they
disagree. 83/83 patterns correspond, 64 byte-identical, 6 differ only by
escaping and are proven equivalent by string identity, and 13 behave
differently — with a witness input for each and misses in both directions.
Cause is two different ReDoS mitigations of the same table: the Node side
bounds the run ({0,256}), the guard excludes the anchor character ([^><]).
Each has a recall cost the other does not.

No data file changed. Behaviour preservation is a v0.1.0 invariant, so the
divergence is reported to the owning repositories, not fixed here. The 8
hybrid severities the guard's port assigns are deliberately NOT copied in —
a second-hand port is not the producing module.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 21:40:26 +02:00
commit 22d3a78591
4 changed files with 193 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -61,6 +61,7 @@ and keeping it that small is the point.
| `spec/decode-pipeline.md` | **Planned, not in v0.1.0.** The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input. Writing it needs the decode implementation, which is engine code and has not been supplied — and a normative spec guessed from a data dump would be worse than an absent one. |
| `conformance/` | **Planned, not in v0.1.0.** One directory per case: `input.txt` in, `expected.json` out. Ground truth. |
| [`docs/extraction-plan.md`](docs/extraction-plan.md) | Informative: where each file was seeded from, and what v0.1.0 promised. |
| [`docs/lexicon-port-divergence.md`](docs/lexicon-port-divergence.md) | Informative: a measured disagreement between two ports of the injection lexicon — 13 patterns that behave differently, in both directions, and why no data file was changed because of it. |
Every JSON file carries a top-level `version`. Every normative specification carries a
`Status: normative` marker. Rows marked **Planned** are named here because the layout is