open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

docs: unlink the three files that do not exist yet, correct the changelog

Browse source 
repo-standard's gate flagged 10 dead internal README links. Seven now
resolve because the files landed this session. The remaining three are
unlinked and marked "Planned, not in v0.1.0" rather than invented:

- signatures/malware-signatures.json — seed data not delivered
- spec/decode-pipeline.md — needs the decode implementation, which is engine
  code and was not supplied. A normative spec guessed from a data dump would
  be worse than an absent one: it would carry a Status: normative marker and
  pass every convention check while saying nothing anyone verified.
- conformance/ — corpus form is still an open decision

The changelog claimed 0.1.0 shipped "a conformance corpus". It does not.
Rewritten to enumerate what is actually present, to state which files were
proven differentially and which one (calibration) carries verified: false,
and to name what is not included. Moved back under [Unreleased] because the
tag has not been cut.

README also now tells a reader that each data file records its own provenance
and verification strength, and which file currently says false.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 21:15:54 +02:00
commit 4742c067a8
2 changed files with 51 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

44
CHANGELOG.md
View file

@ -11,7 +11,45 @@ case ids, disposition semantics). Each JSON file additionally carries its own
## [Unreleased]
## [0.1.0] — 2026-08-09
Initial extraction, in progress. Runtime-neutral detection data and the finding contract,
extracted from the `llm-security` Node implementation and a Python guard without behaviour
change. **Not yet tagged** — see *Not included* below.
Initial extraction. Runtime-neutral detection data, normative contracts and a conformance
corpus, extracted from the `llm-security` Node implementation without behaviour change.
### Added
- `schema/finding.schema.json` — the finding contract plus the SARIF output profile.
Normative. The JSONL profile is deliberately left `unspecified`.
- `signatures/active-content.json` — the EchoLeak class (CVE-2025-32711): 17 patterns,
severities, opacity floors and pass order, from the Python guard.
- `lexicon/injection-lexicon.json` — 83 prompt-injection patterns in four families
(21 critical, 32 high, 22 medium, 8 hybrid).
- `codepoints/carriers.json` — six carrier tables: zero-width characters, the Unicode Tags
block, the Supplementary Private Use Areas, BIDI controls, the Cyrillic presence set and
the 28-entry fold-to-Latin homoglyph map.
- `signatures/secret-egress.json` — the 18 fixed credential and token shapes. Array order
is normative.
- `mapping/owasp-map.json` — four taxonomy maps (LLM, ASI, AST, MCP) over one shared
16-prefix key set.
- `calibration/calibration.json` — risk-score tier constants, verdict thresholds, risk-band
cutoffs, posture grade thresholds.
### Verification
Every file above except `calibration.json` was proven rather than transcribed: the data was
rebuilt **from the commons JSON alone** and diffed against the source implementation. Each
file records its own result and its own limits.
`calibration/calibration.json` carries `verified: false`. Its source arrived as a prose
summary rather than as code, so no differential check was possible, and the file names the
checks that were not run instead of attaching a caveat to a pass.
### Not included
- `signatures/malware-signatures.json` — seed data not yet delivered.
- `spec/decode-pipeline.md` — needs the decode implementation. A normative spec inferred
from a data dump would be worse than an absent one.
- `conformance/` — the corpus form is an open decision: the Python guard's coverage matrix
is probe-based rather than a static input/expected table.
These are named in the README as planned rather than linked, so nothing in the repository
points at a file that does not exist.

14
README.md
View file

@ -53,17 +53,23 @@ and keeping it that small is the point.
| [`lexicon/injection-lexicon.json`](lexicon/injection-lexicon.json) | Prompt-injection pattern lexicon: instruction-override, exfiltration and role-confusion families with per-pattern ids. |
| [`codepoints/carriers.json`](codepoints/carriers.json) | Invisible and deceptive carriers: zero-width characters, BIDI controls, Unicode Tag block ranges, and the homoglyph map. |
| [`signatures/secret-egress.json`](signatures/secret-egress.json) | Credential and token shapes that must never leave a machine, in a portable regex dialect. |
| [`signatures/malware-signatures.json`](signatures/malware-signatures.json) | Signature set for the malicious-code class (`SIG`). |
| `signatures/malware-signatures.json` | **Planned, not in v0.1.0.** Signature set for the malicious-code class (`SIG`). The seed data has not been delivered yet. |
| [`signatures/active-content.json`](signatures/active-content.json) | Active content that renders or fetches on its own — Markdown images, links, reference definitions and autolinks, `data:` URIs, active HTML. The EchoLeak class. |
| [`calibration/calibration.json`](calibration/calibration.json) | The numbers a detector must not invent: risk-score tier constants, verdict thresholds, risk-band cutoffs, posture grade thresholds. Transcribed from a prose summary, not differentially verified — the file says so itself. |
| [`mapping/owasp-map.json`](mapping/owasp-map.json) | Finding-id prefix → OWASP taxonomy entry (LLM / ASI / AST / MCP). |
| [`schema/finding.schema.json`](schema/finding.schema.json) | **Normative.** The finding contract, plus the SARIF and JSONL output profiles. |
| [`spec/decode-pipeline.md`](spec/decode-pipeline.md) | **Normative.** The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input; this is the file that stops that. |
| [`conformance/`](conformance/) | One directory per case: `input.txt` in, `expected.json` out. Ground truth. |
| `spec/decode-pipeline.md` | **Planned, not in v0.1.0.** The decode order, in RFC 2119 language. Two runtimes that decode in different orders will disagree on identical input. Writing it needs the decode implementation, which is engine code and has not been supplied — and a normative spec guessed from a data dump would be worse than an absent one. |
| `conformance/` | **Planned, not in v0.1.0.** One directory per case: `input.txt` in, `expected.json` out. Ground truth. |
| [`docs/extraction-plan.md`](docs/extraction-plan.md) | Informative: where each file was seeded from, and what v0.1.0 promised. |
Every JSON file carries a top-level `version`. Every normative specification carries a
`Status: normative` marker.
`Status: normative` marker. Rows marked **Planned** are named here because the layout is
part of the contract, but the file does not exist yet — they are not links, and nothing in
v0.1.0 depends on them.
Each data file records its own provenance and, in `verified`, how strongly it is backed.
`calibration/calibration.json` is currently the one file that says `false`: it was
transcribed from a prose summary rather than diffed against a running implementation.
### How a consumer proves it conforms