Kjell Tore Guttormsen
0433240b14
The finding contract plus the SARIF 2.1.0 output profile, modelled on llm-security's sarif-formatter.mjs (operator dump). Draft 2020-12, valid against the metaschema. The SARIF profile is proven, not transcribed: the mapping was re-implemented from the commons JSON alone and diffed against the real toSARIF over 10 envelope shapes (all severities incl. unknown/undefined, slug edge cases, rule-id collision, every optional-field combination, multi-scanner, explicit version) — 0 differences. Three things left honest rather than closed: - additionalProperties stays open. The formatter CONSUMES findings, so fields it ignores are invisible in this evidence; the property list is a lower bound until the producer is supplied. - The JSONL profile is status=unspecified with a TODO. "One finding per line" is inference, and a guessed normative contract is worse than a missing one. - Node and the Python guard share only severity+owasp; the rest of the field names diverge (scanner/title vs detector/label). Recorded as an open question for v0.2.0, not settled silently here. One published claim was wrong and is corrected: rule-id collision does NOT cover punctuation, only case and whitespace runs. Caught by executing the claim instead of asserting it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0191AKc2qW6tmXDFSx1xn53q
124 lines
8.1 KiB
Markdown
124 lines
8.1 KiB
Markdown
# Extraction plan — v0.1.0
|
||
|
||
**Status: informative.** This is the plan of record for how this repository came to exist,
|
||
copied verbatim (structure preserved, lightly reformatted) from the operator brief that
|
||
opened it. It is **not** normative: nothing here constrains a consumer. When it disagrees
|
||
with `spec/` or `schema/`, those win.
|
||
|
||
Origin: **Phase 4 of the `llm-security` v8 plan**, which lives in the sibling repository
|
||
`llm-security`. That repository is context only — no session in this repository reads from
|
||
or writes to it.
|
||
|
||
## Charter
|
||
|
||
No engine code. Only: JSON data, normative specs, and a conformance corpus that several
|
||
runtimes (Node in `llm-security`, Python in a guard repo, a wiki) can run against and get
|
||
an identical verdict from. The pattern is copied from the sibling repository
|
||
`portfolio-optimiser-commons` (hard charter: "nothing here may import/depend on a
|
||
framework").
|
||
|
||
## Layout
|
||
|
||
```
|
||
llm-security-commons/
|
||
README.md # charter: data+contract+fixtures only, no engine code
|
||
lexicon/injection-lexicon.json
|
||
codepoints/carriers.json # zero-width, BIDI, Unicode-Tag ranges, homoglyph map
|
||
signatures/secret-egress.json
|
||
signatures/malware-signatures.json
|
||
signatures/active-content.json # EchoLeak: MD image/link/refdef/autolink, data:, active HTML
|
||
calibration/calibration.json # entropy floors, scan caps, disposition ranks
|
||
mapping/owasp-map.json # prefix -> LLM/ASI/AST/MCP
|
||
schema/finding.schema.json # + SARIF & JSONL profiles. Status: normative
|
||
spec/decode-pipeline.md # normative RFC-2119 decode order
|
||
conformance/ # {case}/input.txt + {case}/expected.json
|
||
STATE.md # LOCAL-ONLY / gitignored (mirror commons convention)
|
||
```
|
||
|
||
Every JSON file carries a top-level `"version"` field. Every spec carries a
|
||
`Status: normative` marker.
|
||
|
||
## v0.1.0 seed sources
|
||
|
||
`llm-security` is the canonical and richest source. **This repository's sessions have no
|
||
read access to it** — content arrives only as an operator-supplied dump. Security-critical
|
||
tables (homoglyph map, secret patterns, malware signatures) MUST come from real source
|
||
data, never from recollection or inference.
|
||
|
||
| Target | Seed source in `llm-security` (unless noted) |
|
||
|---|---|
|
||
| `lexicon/injection-lexicon.json` | `scanners/lib/injection-patterns.mjs` |
|
||
| `codepoints/carriers.json` | `scanners/unicode-scanner.mjs` + `scanners/lib/string-utils.mjs` (incl. `HOMOGLYPH_MAP`) |
|
||
| `signatures/secret-egress.json` | `knowledge/secrets-patterns.md` — the **18-entry hook table**, NOT the PCRE-flavored agent-consumed variant |
|
||
| `signatures/malware-signatures.json` | `knowledge/signatures.json` (the SIG scanner) |
|
||
| `signatures/active-content.json` | currently only in a guard repo's `active_content.py`. If unavailable: stub with a version field and a TODO naming the source |
|
||
| `calibration/calibration.json` | `scanners/lib/severity.mjs` — thresholds + scanner caps |
|
||
| `mapping/owasp-map.json` | `scanners/lib/severity.mjs` — `OWASP_MAP` (+ 3 sibling maps in the same file) |
|
||
| `schema/finding.schema.json` | modelled on `scanners/lib/sarif-formatter.mjs`'s SARIF shape |
|
||
| `conformance/` | union of the guard repo's `coverage.py` matrix (126 classes + 4 gaps-must-hold) and `llm-security/examples/` |
|
||
|
||
## Constraints
|
||
|
||
- Offline / deterministic only — no network, no model calls inside the data itself.
|
||
- Forgejo `open/` — never GitHub.
|
||
- MIT license, fork-and-own.
|
||
- `STATE.md` is LOCAL-ONLY (gitignored) — same convention as the rest of the polyrepo.
|
||
- **Behaviour preservation is the point:** this must not change a single finding in
|
||
`llm-security` when it is later consumed from here. That consumption happens in
|
||
`llm-security`'s own Phase 5 steps 3–4 — **not here.**
|
||
|
||
## Verification log
|
||
|
||
Every claim of fidelity below was produced by a command, not by reading. The check scripts
|
||
themselves deliberately do **not** live in this repository — executable code here would
|
||
breach the charter. They are reproducible from the description given.
|
||
|
||
### `signatures/active-content.json` — extracted 2026-08-09
|
||
|
||
Source: `llm-ingestion-pipeline-security` v0.3.4, commit `0bf0729` (2026-08-03),
|
||
`src/llm_ingestion_guard/active_content.py` + `calibration.py`. Read-only; nothing in that
|
||
repository was modified.
|
||
|
||
| Check | Method | Result |
|
||
| --- | --- | --- |
|
||
| JSON well-formed | `python3 -m json.tool` | pass |
|
||
| Patterns compile as Python `re` | translate `(?<` → `(?P<`, compile all 17 with declared flags | 17/17, 0 failures |
|
||
| Patterns compile as ECMAScript | `new RegExp(pattern, flags)` on all 17 | 17/17, 0 failures |
|
||
| Pattern text matches source | compare against the live `re.Pattern.pattern` of each source object, inline flags stripped | 12/17 byte-identical; 5 differ only by the documented `redundant-quote-escape` normalisation |
|
||
| The 5 normalised patterns behave identically | differential match-set comparison (offsets + captured text) against the source objects over a 30-input adversarial corpus: bare quotes, escaped quotes, markdown titles containing quotes, quoted/unquoted HTML attributes, quote runs of length 1–5 | 150 comparisons, 0 differences |
|
||
| The normalisation is necessary | `new RegExp('\\"', 'u')` and `'v'` in Node | both throw `Invalid escape`; the bare form compiles under `""`, `"u"` and `"v"` |
|
||
| Severities, ordinary severity, opacity floors, active-tag set, pass order | compare against `calibration.ACTIVE_CONTENT_SEVERITY`, `ACTIVE_CONTENT_ORDINARY_SEVERITY`, `URL_OPAQUE_*`, `active_content._ACTIVE_TAGS`, and the scan-call order in `scan_active_content` | all identical (23/23 tags, 6/6 severities, 4/4 floors) |
|
||
|
||
Not verified, and not claimed: that the Node consumer's active-content behaviour matches
|
||
this table. The source module states the Node port shares its severities; that is the
|
||
module's claim, and confirming it needs the Node file.
|
||
|
||
### `schema/finding.schema.json` — extracted 2026-08-09
|
||
|
||
Source: `llm-security/scanners/lib/sarif-formatter.mjs`, supplied as an operator dump. No
|
||
commit hash accompanied it, so provenance is recorded as `unknown` rather than guessed.
|
||
|
||
| Check | Method | Result |
|
||
| --- | --- | --- |
|
||
| JSON well-formed | `python3 -m json.tool` | pass |
|
||
| Valid JSON Schema | `jsonschema` `check_schema` against draft 2020-12 | pass |
|
||
| Accepts/rejects findings correctly | 2 valid + 3 invalid findings (missing `scanner`, unknown severity, `line: 0`) | 5/5 as intended |
|
||
| SARIF profile reproduces the source | re-implemented the mapping **from the commons JSON alone** and diffed `JSON.stringify` against the real `toSARIF` over 10 envelope shapes: empty, missing `scanners`, empty `scanners`, scanner with no findings, all five severities plus an unknown and an `undefined` one, five slug edge cases (double space, tab, newline, leading/trailing space, mixed case), a rule-id collision, all seven optional-field combinations, two scanners, and an explicit `version` argument | 10/10 identical, 0 differences |
|
||
| The three `known_lossiness` claims are true | executed each against the real formatter | all three confirmed, **and one earlier claim corrected**: punctuation does *not* collapse — the slug lowercases and collapses whitespace only, so `Zero-width carrier` and `Zero-width carrier!` remain distinct ids. The wrong claim was published in the first draft of this file and fixed before commit. |
|
||
|
||
Not verified, and recorded in the file as open: the finding **producer** was not supplied, so
|
||
the property list is a lower bound; `scanner` and `severity` are required by design rather
|
||
than by evidence; and the JSONL profile is left explicitly `unspecified` rather than
|
||
invented, because "one finding per line" is inference.
|
||
|
||
## Definition of done for v0.1.0
|
||
|
||
1. Repository initialized, Forgejo remote `open/llm-security-commons`, MIT, `STATE.md`
|
||
gitignored.
|
||
2. Every file in the layout above present and populated from verified seed data — or
|
||
explicitly and visibly stubbed where the source was unavailable.
|
||
3. All JSON well-formed, every data file carrying `"version"`, every spec carrying
|
||
`Status: normative`.
|
||
4. Tagged `v0.1.0` and pushed.
|
||
5. A `coord` message sent to `llm-security` announcing that the repository and `v0.1.0`
|
||
exist, so Phase 5 step 3 (vendoring) can start from there.
|