llm-security-commons/docs/extraction-plan.md

Extraction plan — v0.1.0

Status: informative. This is the plan of record for how this repository came to exist, copied verbatim (structure preserved, lightly reformatted) from the operator brief that opened it. It is not normative: nothing here constrains a consumer. When it disagrees with spec/ or schema/, those win.

Origin: Phase 4 of the llm-security v8 plan, which lives in the sibling repository llm-security. That repository is context only — no session in this repository reads from or writes to it.

Charter

No engine code. Only: JSON data, normative specs, and a conformance corpus that several runtimes (Node in llm-security, Python in a guard repo, a wiki) can run against and get an identical verdict from. The pattern is copied from the sibling repository portfolio-optimiser-commons (hard charter: "nothing here may import/depend on a framework").

Layout

llm-security-commons/
  README.md                       # charter: data+contract+fixtures only, no engine code
  lexicon/injection-lexicon.json
  codepoints/carriers.json        # zero-width, BIDI, Unicode-Tag ranges, homoglyph map
  signatures/secret-egress.json
  signatures/malware-signatures.json
  signatures/active-content.json  # EchoLeak: MD image/link/refdef/autolink, data:, active HTML
  calibration/calibration.json    # entropy floors, scan caps, disposition ranks
  mapping/owasp-map.json          # prefix -> LLM/ASI/AST/MCP
  schema/finding.schema.json      # + SARIF & JSONL profiles. Status: normative
  spec/decode-pipeline.md         # normative RFC-2119 decode order
  conformance/                    # {case}/input.txt + {case}/expected.json
  STATE.md                        # LOCAL-ONLY / gitignored (mirror commons convention)

Every JSON file carries a top-level "version" field. Every spec carries a Status: normative marker.

v0.1.0 seed sources

llm-security is the canonical and richest source. This repository's sessions have no read access to it — content arrives only as an operator-supplied dump. Security-critical tables (homoglyph map, secret patterns, malware signatures) MUST come from real source data, never from recollection or inference.

Target	Seed source in llm-security (unless noted)
lexicon/injection-lexicon.json	scanners/lib/injection-patterns.mjs
codepoints/carriers.json	scanners/unicode-scanner.mjs + scanners/lib/string-utils.mjs (incl. HOMOGLYPH_MAP)
signatures/secret-egress.json	knowledge/secrets-patterns.md — the 18-entry hook table, NOT the PCRE-flavored agent-consumed variant
signatures/malware-signatures.json	knowledge/signatures.json (the SIG scanner)
signatures/active-content.json	currently only in a guard repo's active_content.py. If unavailable: stub with a version field and a TODO naming the source
calibration/calibration.json	scanners/lib/severity.mjs — thresholds + scanner caps
mapping/owasp-map.json	scanners/lib/severity.mjs — OWASP_MAP (+ 3 sibling maps in the same file)
schema/finding.schema.json	modelled on scanners/lib/sarif-formatter.mjs's SARIF shape
conformance/	union of the guard repo's coverage.py matrix (126 classes + 4 gaps-must-hold) and llm-security/examples/

Constraints

• Offline / deterministic only — no network, no model calls inside the data itself.
• Forgejo open/ — never GitHub.
• MIT license, fork-and-own.
• STATE.md is LOCAL-ONLY (gitignored) — same convention as the rest of the polyrepo.
• Behaviour preservation is the point: this must not change a single finding in llm-security when it is later consumed from here. That consumption happens in llm-security's own Phase 5 steps 3–4 — not here.

Definition of done for v0.1.0

1. Repository initialized, Forgejo remote open/llm-security-commons, MIT, STATE.md gitignored.
2. Every file in the layout above present and populated from verified seed data — or explicitly and visibly stubbed where the source was unavailable.
3. All JSON well-formed, every data file carrying "version", every spec carrying Status: normative.
4. Tagged v0.1.0 and pushed.
5. A coord message sent to llm-security announcing that the repository and v0.1.0 exist, so Phase 5 step 3 (vendoring) can start from there.