feat: initialize llm-security-commons (charter, license, extraction plan)
Data+contract+fixtures repository for runtime-neutral LLM/agent security detection. Charter is hard: no engine code, no dependencies, no runtime — JSON data, normative specs and conformance fixtures only. Scaffolding only; the seed data files land in follow-up commits from verified source dumps (never from recollection). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0191AKc2qW6tmXDFSx1xn53q
This commit is contained in:
commit
ef80288c57
6 changed files with 385 additions and 0 deletions
113
README.md
Normal file
113
README.md
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
# llm-security-commons
|
||||
|
||||
Runtime-neutral core for LLM and agent security detection: detector data, normative contracts and a conformance corpus that several runtimes can share.
|
||||
|
||||
[](LICENSE)
|
||||
|
||||
Detection logic gets reimplemented every time it crosses a language boundary, and the
|
||||
copies drift: the Node scanner flags a zero-width carrier the Python guard misses, and
|
||||
nobody notices until an incident. This repository holds the part that should never have
|
||||
been copied — the pattern tables, the code-point carriers, the calibration thresholds, the
|
||||
finding contract, and a fixture corpus with expected verdicts — so that two independent
|
||||
implementations can be held to the same answer on the same input.
|
||||
|
||||
It is for anyone building or maintaining a detector for prompt injection, secret egress,
|
||||
unicode-carrier smuggling or active content in untrusted text, on any runtime.
|
||||
|
||||
**It holds no runnable code.** Data, specifications and fixtures only.
|
||||
|
||||
## Install
|
||||
|
||||
Nothing to install — this repository is **vendored into consumers**, not installed.
|
||||
|
||||
As a `git subtree` (recommended: history is preserved and upgrades are a single command):
|
||||
|
||||
```bash
|
||||
git subtree add --prefix vendor/commons \
|
||||
https://git.fromaitochitta.com/open/llm-security-commons.git v0.1.0 --squash
|
||||
|
||||
# later, to move to a newer tag
|
||||
git subtree pull --prefix vendor/commons \
|
||||
https://git.fromaitochitta.com/open/llm-security-commons.git v0.2.0 --squash
|
||||
```
|
||||
|
||||
Or pin a tag and copy — `fork-and-own` is an explicitly supported path:
|
||||
|
||||
```bash
|
||||
git clone --depth 1 --branch v0.1.0 \
|
||||
https://git.fromaitochitta.com/open/llm-security-commons.git
|
||||
```
|
||||
|
||||
Always vendor **a tag**, never `main`. The tag is what a conformance result can be
|
||||
attributed to.
|
||||
|
||||
## Requirements
|
||||
|
||||
A JSON parser and the ability to read a text file. That is the entire dependency surface,
|
||||
and keeping it that small is the point.
|
||||
|
||||
## What it does
|
||||
|
||||
| Path | Contents |
|
||||
| --- | --- |
|
||||
| [`lexicon/injection-lexicon.json`](lexicon/injection-lexicon.json) | Prompt-injection pattern lexicon: instruction-override, exfiltration and role-confusion families with per-pattern ids. |
|
||||
| [`codepoints/carriers.json`](codepoints/carriers.json) | Invisible and deceptive carriers: zero-width characters, BIDI controls, Unicode Tag block ranges, and the homoglyph map. |
|
||||
| [`signatures/secret-egress.json`](signatures/secret-egress.json) | Credential and token shapes that must never leave a machine, in a portable regex dialect. |
|
||||
| [`signatures/malware-signatures.json`](signatures/malware-signatures.json) | Signature set for the malicious-code class (`SIG`). |
|
||||
| [`signatures/active-content.json`](signatures/active-content.json) | Active content that renders or fetches on its own — Markdown images, links, reference definitions and autolinks, `data:` URIs, active HTML. The EchoLeak class. |
|
||||
| [`calibration/calibration.json`](calibration/calibration.json) | The numbers a detector must not invent: entropy floors, scan caps, disposition ranks. |
|
||||
| [`mapping/owasp-map.json`](mapping/owasp-map.json) | Finding-id prefix → OWASP taxonomy entry (LLM / ASI / AST / MCP). |
|
||||
| [`schema/finding.schema.json`](schema/finding.schema.json) | **Normative.** The finding contract, plus the SARIF and JSONL output profiles. |
|
||||
| [`spec/decode-pipeline.md`](spec/decode-pipeline.md) | **Normative.** The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input; this is the file that stops that. |
|
||||
| [`conformance/`](conformance/) | One directory per case: `input.txt` in, `expected.json` out. Ground truth. |
|
||||
| [`docs/extraction-plan.md`](docs/extraction-plan.md) | Informative: where each file was seeded from, and what v0.1.0 promised. |
|
||||
|
||||
Every JSON file carries a top-level `version`. Every normative specification carries a
|
||||
`Status: normative` marker.
|
||||
|
||||
### How a consumer proves it conforms
|
||||
|
||||
Run every `conformance/<case>/input.txt` through your detector, serialize the result per
|
||||
[`schema/finding.schema.json`](schema/finding.schema.json), and compare to
|
||||
`expected.json`. Disagreement means your runtime is wrong, or the fixture is — and the
|
||||
fixture only changes in its own commit, with the reason written down.
|
||||
|
||||
There is **no CI in this organisation** and nothing runs that comparison automatically. It
|
||||
runs in each consumer's own test suite, against a pinned tag.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Not a scanner.** There is no engine here, and there will not be one. If you are looking
|
||||
for something to run, you want a consumer — `llm-security` for Claude Code.
|
||||
- **Not a framework or a library.** No package manifest, no dependencies, no build.
|
||||
- **Not a general-purpose Unicode or regex toolkit.** The tables cover what the detection
|
||||
classes need, not the standard.
|
||||
- **Not a vulnerability feed.** No CVEs, no advisories, nothing time-sensitive. Everything
|
||||
here is offline and deterministic.
|
||||
- **Not a policy engine.** `calibration.json` publishes the thresholds; deciding what to do
|
||||
when one is crossed belongs to the consumer.
|
||||
- **Not the place to fix a consumer's behaviour.** Data extracted from an implementation is
|
||||
kept behaviour-identical on purpose. A disagreement is reported to that implementation
|
||||
and decided there, where it is tested.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- **Coverage is the union of what the seed implementations detected**, not of what exists.
|
||||
A class absent from `conformance/` has not been shown to work anywhere.
|
||||
- **Regex portability is a real risk.** Pattern data is written for a common subset, but
|
||||
engines differ (lookbehind, named groups, Unicode property escapes). A consumer whose
|
||||
engine rejects a pattern must report it rather than silently skip it — a skipped pattern
|
||||
is an invisible false negative.
|
||||
- **Fixtures prove agreement, not correctness.** Two runtimes passing the same corpus agree
|
||||
with each other and with the fixture author. A wrong `expected.json` makes both wrong
|
||||
identically.
|
||||
- **The homoglyph map is finite.** Confusable coverage is a long tail; absence from the map
|
||||
is not evidence a character is safe.
|
||||
|
||||
## Changelog
|
||||
|
||||
See [CHANGELOG.md](CHANGELOG.md).
|
||||
|
||||
## License
|
||||
|
||||
MIT — see [LICENSE](LICENSE). Fork-and-own is an intended use, not a tolerated one.
|
||||
Loading…
Add table
Add a link
Reference in a new issue