Kjell Tore Guttormsen
ff18768b71
Four corrections across the two informative documents, all of them to
claims this repository made and none of them to a data file.
The retraction is the important one. lexicon-port-divergence.md said
neither runtime misses an attack: every witness payload still produced a
finding, so the divergence was only in which labels were raised. That was
wrong in method rather than in arithmetic. The payloads had been run
against the UNION of every pattern table this repository holds - 111 rules
across the lexicon, active-content.json and secret-egress.json - and the
hit that rescued them came from active-content.json, which is the Python
guard's table. llm-security has no active-content table at all. Unioning
two runtimes' tables and reading the result as a statement about each of
them turns combined reach into individual coverage.
Re-measured through llm-security's own entry point scanForInjection() at
b0de0ca - the whole engine, normalisation, homoglyph folding, rot13 and all
four arrays - all three witness payloads return found: false. Not a
different label set: no finding at all. Controls in the same run behave
normally, so the {0,256} window is a real evasion window and the <script>
pattern really does require a closing tag. llm-security reached the same
three results independently and has logged them as a v8.x task.
Second: one of the 13 divergences does not reach report level. On the
iframe witness the guard misses hybrid-xss:iframe-src but its
hybrid-xss:javascript-uri fires at the same severity and the same OWASP
anchor, so a reader of its report still sees the payload flagged. Both
sides were re-measured to confirm. The report-level number is 12; the 13
stands as the number that blocks conformance/, because a fixture names
labels.
Third: extraction-plan.md said six of the confusables have no entry in the
fold map. It is zero - all 13 are keys in the 28-entry map, and
carriers.json has said so correctly since it was written. The prose, not
the data, carried the error, and the number six most likely came from
running the comparison the other way (five fold-map Cyrillic keys are not
confusables). It also called the diverging sets deliberately distinct;
llm-security states the U+0456 / U+0445 divergence is drift, not design,
and has logged it for v8.x. Those two readings imply opposite things about
whether the sets should be expected to converge.
Fourth: the seed-sources section still asserted that sessions here have no
read access to llm-security and that content arrives only as a dump. The
operator changed that on 2026-08-09. It now records the read-only pinned-
commit boundary, the still-absolute prohibition on the sibling working copy
and on writing, and the reason the change matters - four attribution
defects caught so far, every one a transcription artefact.
Status blocks updated: all eight JSON artefacts now exist, seven verified
differentially and calibration.json still honestly verified: false; the
conformance blocker is half-cleared, with naming ratified and the 13
divergent patterns still unresolved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SNMcqrfNyoLRQ7qXUFZnb9
|
||
|---|---|---|
| calibration | ||
| codepoints | ||
| docs | ||
| lexicon | ||
| mapping | ||
| schema | ||
| signatures | ||
| .gitignore | ||
| CHANGELOG.md | ||
| CLAUDE.md | ||
| LICENSE | ||
| README.md | ||
llm-security-commons
Runtime-neutral core for LLM and agent security detection: detector data, normative contracts and a conformance corpus that several runtimes can share.
Detection logic gets reimplemented every time it crosses a language boundary, and the copies drift: the Node scanner flags a zero-width carrier the Python guard misses, and nobody notices until an incident. This repository holds the part that should never have been copied — the pattern tables, the code-point carriers, the calibration thresholds, the finding contract, and a fixture corpus with expected verdicts — so that two independent implementations can be held to the same answer on the same input.
It is for anyone building or maintaining a detector for prompt injection, secret egress, unicode-carrier smuggling or active content in untrusted text, on any runtime.
It holds no runnable code. Data, specifications and fixtures only.
Install
Nothing to install — this repository is vendored into consumers, not installed.
As a git subtree (recommended: history is preserved and upgrades are a single command):
git subtree add --prefix vendor/commons \
https://git.fromaitochitta.com/open/llm-security-commons.git v0.1.0 --squash
# later, to move to a newer tag
git subtree pull --prefix vendor/commons \
https://git.fromaitochitta.com/open/llm-security-commons.git v0.2.0 --squash
Or pin a tag and copy — fork-and-own is an explicitly supported path:
git clone --depth 1 --branch v0.1.0 \
https://git.fromaitochitta.com/open/llm-security-commons.git
Always vendor a tag, never main. The tag is what a conformance result can be
attributed to.
Requirements
A JSON parser and the ability to read a text file. That is the entire dependency surface, and keeping it that small is the point.
What it does
| Path | Contents |
|---|---|
lexicon/injection-lexicon.json |
Prompt-injection pattern lexicon: 83 patterns in four severity families (critical, high, medium, hybrid), each with a stable id and per-runtime aliases. The thematic class (override:, evasion:, hitl-trap:, …) is the id prefix, not the family. |
codepoints/carriers.json |
Invisible and deceptive carriers: zero-width characters, BIDI controls, Unicode Tag block ranges, and the homoglyph map. |
signatures/secret-egress.json |
Credential and token shapes that must never leave a machine, in a portable regex dialect. |
signatures/malware-signatures.json |
Known-bad identity for the malicious-code class (SIG): seven tight signatures over four families — PHP webshells, reverse shells, cryptominers, offensive tooling. Seven signatures are not malware coverage, and the file says so. |
signatures/active-content.json |
Active content that renders or fetches on its own — Markdown images, links, reference definitions and autolinks, data: URIs, active HTML. The EchoLeak class. |
calibration/calibration.json |
The numbers a detector must not invent: risk-score tier constants, verdict thresholds, risk-band cutoffs, posture grade thresholds. Transcribed from a prose summary, not differentially verified — the file says so itself. |
mapping/owasp-map.json |
Finding-id prefix → OWASP taxonomy entry (LLM / ASI / AST / MCP). |
schema/finding.schema.json |
Normative. The finding contract — closed against its producer, ten properties — plus the SARIF output profile. The JSONL profile is recorded as not applicable, with the reason. |
spec/decode-pipeline.md |
Planned, not in v0.1.0. The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input. Writing it needs the decode implementation, which is engine code and has not been supplied — and a normative spec guessed from a data dump would be worse than an absent one. |
conformance/ |
Planned, not in v0.1.0. One directory per case: input.txt in, expected.json out. Ground truth. |
docs/extraction-plan.md |
Informative: where each file was seeded from, and what v0.1.0 promised. |
docs/lexicon-port-divergence.md |
Informative: a measured disagreement between two ports of the injection lexicon — 13 patterns that behave differently, in both directions, and why no data file was changed because of it. |
Every JSON file carries a top-level version. Every normative specification carries a
Status: normative marker. Rows marked Planned are named here because the layout is
part of the contract, but the file does not exist yet — they are not links, and nothing in
v0.1.0 depends on them.
Each data file records its own provenance and, in verified, how strongly it is backed.
calibration/calibration.json is currently the one file that says false: it was
transcribed from a prose summary rather than diffed against a running implementation.
How a consumer proves it conforms
Run every conformance/<case>/input.txt through your detector, serialize the result per
schema/finding.schema.json, and compare to
expected.json. Disagreement means your runtime is wrong, or the fixture is — and the
fixture only changes in its own commit, with the reason written down.
There is no CI in this organisation and nothing runs that comparison automatically. It runs in each consumer's own test suite, against a pinned tag.
Non-goals
- Not a scanner. There is no engine here, and there will not be one. If you are looking
for something to run, you want a consumer —
llm-securityfor Claude Code. - Not a framework or a library. No package manifest, no dependencies, no build.
- Not a general-purpose Unicode or regex toolkit. The tables cover what the detection classes need, not the standard.
- Not a vulnerability feed. No CVEs, no advisories, nothing time-sensitive. Everything here is offline and deterministic.
- Not a policy engine.
calibration.jsonpublishes the thresholds; deciding what to do when one is crossed belongs to the consumer. - Not the place to fix a consumer's behaviour. Data extracted from an implementation is kept behaviour-identical on purpose. A disagreement is reported to that implementation and decided there, where it is tested.
Known limitations
- Coverage is the union of what the seed implementations detected, not of what exists.
A class absent from
conformance/has not been shown to work anywhere. - Regex portability is a real risk. Pattern data is written for a common subset, but engines differ (lookbehind, named groups, Unicode property escapes). A consumer whose engine rejects a pattern must report it rather than silently skip it — a skipped pattern is an invisible false negative.
- Fixtures prove agreement, not correctness. Two runtimes passing the same corpus agree
with each other and with the fixture author. A wrong
expected.jsonmakes both wrong identically. - The homoglyph map is finite. Confusable coverage is a long tail; absence from the map is not evidence a character is safe.
Changelog
See CHANGELOG.md.
License
MIT — see LICENSE. Fork-and-own is an intended use, not a tolerated one.