Kjell Tore Guttormsen
8ee297c585
The risk-score tier constants, verdict thresholds, risk-band cutoffs and posture grade thresholds. Constants only — the log scaling and the if/else chains that consume them are engine and stay in the consumer. This file is the exception in this repository and is marked as such in its own verification block. Every other data file here was rebuilt from its commons JSON and diffed against an imported module. This one arrived as a human-written PROSE SUMMARY of severity.mjs, so there was nothing to import and nothing to diff. verified: false, with the two skipped checks named rather than a caveat attached to a pass. The consequence inverts this repository's central rule and the file says so: for every other file a disagreeing consumer is wrong; for this one, until the module arrives in executable form, a disagreement is not automatically the consumer's bug. What COULD be checked was: bands are contiguous and non-overlapping across 0-100, the BLOCK trigger (65) is exactly the Critical band floor, the WARNING trigger (15) is exactly the Medium band floor, and the per-tier reachable minima recompute exactly (80/48/20/4, exact because log2(2) is 1). Corrects README and the extraction plan in the same commit: both promised entropy floors, scan caps and disposition ranks. None arrived — 0 occurrences each across the whole dump. Named in the file under not_supplied so the absence is visible rather than inferred, and both rows now describe what is actually present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
113 lines
6.2 KiB
Markdown
113 lines
6.2 KiB
Markdown
# llm-security-commons
|
|
|
|
Runtime-neutral core for LLM and agent security detection: detector data, normative contracts and a conformance corpus that several runtimes can share.
|
|
|
|
[](LICENSE)
|
|
|
|
Detection logic gets reimplemented every time it crosses a language boundary, and the
|
|
copies drift: the Node scanner flags a zero-width carrier the Python guard misses, and
|
|
nobody notices until an incident. This repository holds the part that should never have
|
|
been copied — the pattern tables, the code-point carriers, the calibration thresholds, the
|
|
finding contract, and a fixture corpus with expected verdicts — so that two independent
|
|
implementations can be held to the same answer on the same input.
|
|
|
|
It is for anyone building or maintaining a detector for prompt injection, secret egress,
|
|
unicode-carrier smuggling or active content in untrusted text, on any runtime.
|
|
|
|
**It holds no runnable code.** Data, specifications and fixtures only.
|
|
|
|
## Install
|
|
|
|
Nothing to install — this repository is **vendored into consumers**, not installed.
|
|
|
|
As a `git subtree` (recommended: history is preserved and upgrades are a single command):
|
|
|
|
```bash
|
|
git subtree add --prefix vendor/commons \
|
|
https://git.fromaitochitta.com/open/llm-security-commons.git v0.1.0 --squash
|
|
|
|
# later, to move to a newer tag
|
|
git subtree pull --prefix vendor/commons \
|
|
https://git.fromaitochitta.com/open/llm-security-commons.git v0.2.0 --squash
|
|
```
|
|
|
|
Or pin a tag and copy — `fork-and-own` is an explicitly supported path:
|
|
|
|
```bash
|
|
git clone --depth 1 --branch v0.1.0 \
|
|
https://git.fromaitochitta.com/open/llm-security-commons.git
|
|
```
|
|
|
|
Always vendor **a tag**, never `main`. The tag is what a conformance result can be
|
|
attributed to.
|
|
|
|
## Requirements
|
|
|
|
A JSON parser and the ability to read a text file. That is the entire dependency surface,
|
|
and keeping it that small is the point.
|
|
|
|
## What it does
|
|
|
|
| Path | Contents |
|
|
| --- | --- |
|
|
| [`lexicon/injection-lexicon.json`](lexicon/injection-lexicon.json) | Prompt-injection pattern lexicon: instruction-override, exfiltration and role-confusion families with per-pattern ids. |
|
|
| [`codepoints/carriers.json`](codepoints/carriers.json) | Invisible and deceptive carriers: zero-width characters, BIDI controls, Unicode Tag block ranges, and the homoglyph map. |
|
|
| [`signatures/secret-egress.json`](signatures/secret-egress.json) | Credential and token shapes that must never leave a machine, in a portable regex dialect. |
|
|
| [`signatures/malware-signatures.json`](signatures/malware-signatures.json) | Signature set for the malicious-code class (`SIG`). |
|
|
| [`signatures/active-content.json`](signatures/active-content.json) | Active content that renders or fetches on its own — Markdown images, links, reference definitions and autolinks, `data:` URIs, active HTML. The EchoLeak class. |
|
|
| [`calibration/calibration.json`](calibration/calibration.json) | The numbers a detector must not invent: risk-score tier constants, verdict thresholds, risk-band cutoffs, posture grade thresholds. Transcribed from a prose summary, not differentially verified — the file says so itself. |
|
|
| [`mapping/owasp-map.json`](mapping/owasp-map.json) | Finding-id prefix → OWASP taxonomy entry (LLM / ASI / AST / MCP). |
|
|
| [`schema/finding.schema.json`](schema/finding.schema.json) | **Normative.** The finding contract, plus the SARIF and JSONL output profiles. |
|
|
| [`spec/decode-pipeline.md`](spec/decode-pipeline.md) | **Normative.** The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input; this is the file that stops that. |
|
|
| [`conformance/`](conformance/) | One directory per case: `input.txt` in, `expected.json` out. Ground truth. |
|
|
| [`docs/extraction-plan.md`](docs/extraction-plan.md) | Informative: where each file was seeded from, and what v0.1.0 promised. |
|
|
|
|
Every JSON file carries a top-level `version`. Every normative specification carries a
|
|
`Status: normative` marker.
|
|
|
|
### How a consumer proves it conforms
|
|
|
|
Run every `conformance/<case>/input.txt` through your detector, serialize the result per
|
|
[`schema/finding.schema.json`](schema/finding.schema.json), and compare to
|
|
`expected.json`. Disagreement means your runtime is wrong, or the fixture is — and the
|
|
fixture only changes in its own commit, with the reason written down.
|
|
|
|
There is **no CI in this organisation** and nothing runs that comparison automatically. It
|
|
runs in each consumer's own test suite, against a pinned tag.
|
|
|
|
## Non-goals
|
|
|
|
- **Not a scanner.** There is no engine here, and there will not be one. If you are looking
|
|
for something to run, you want a consumer — `llm-security` for Claude Code.
|
|
- **Not a framework or a library.** No package manifest, no dependencies, no build.
|
|
- **Not a general-purpose Unicode or regex toolkit.** The tables cover what the detection
|
|
classes need, not the standard.
|
|
- **Not a vulnerability feed.** No CVEs, no advisories, nothing time-sensitive. Everything
|
|
here is offline and deterministic.
|
|
- **Not a policy engine.** `calibration.json` publishes the thresholds; deciding what to do
|
|
when one is crossed belongs to the consumer.
|
|
- **Not the place to fix a consumer's behaviour.** Data extracted from an implementation is
|
|
kept behaviour-identical on purpose. A disagreement is reported to that implementation
|
|
and decided there, where it is tested.
|
|
|
|
## Known limitations
|
|
|
|
- **Coverage is the union of what the seed implementations detected**, not of what exists.
|
|
A class absent from `conformance/` has not been shown to work anywhere.
|
|
- **Regex portability is a real risk.** Pattern data is written for a common subset, but
|
|
engines differ (lookbehind, named groups, Unicode property escapes). A consumer whose
|
|
engine rejects a pattern must report it rather than silently skip it — a skipped pattern
|
|
is an invisible false negative.
|
|
- **Fixtures prove agreement, not correctness.** Two runtimes passing the same corpus agree
|
|
with each other and with the fixture author. A wrong `expected.json` makes both wrong
|
|
identically.
|
|
- **The homoglyph map is finite.** Confusable coverage is a long tail; absence from the map
|
|
is not evidence a character is safe.
|
|
|
|
## Changelog
|
|
|
|
See [CHANGELOG.md](CHANGELOG.md).
|
|
|
|
## License
|
|
|
|
MIT — see [LICENSE](LICENSE). Fork-and-own is an intended use, not a tolerated one.
|