feat(signatures): add secret-egress.json from verified llm-security dump
The 18 fixed credential and token shapes a pre-write guard matches before content is persisted: cloud keys, vendor tokens, PEM blocks, connection strings, JWTs. Proven, not transcribed: the table was rebuilt from the commons JSON alone and diffed against the imported dump module — 18/18 identical on name, source and flags, and all 18 byte-identical, so no normalisation was needed. All 18 compile in Node bare, Node under `u`, and Python `re`. Array order is normative and is tested as such, not merely asserted: a Bearer header containing a JWT must be labelled "Authorization header with token" rather than "JWT (three-part token)", which is why the source puts the bare JWT entry last. Reproduced from the commons order, and shown to change under a reversed table. Every entry carries an explicit `order` field so a JSON round-trip cannot reorder the contract silently. Corrects the extraction plan's seed-source row in the same commit: it named knowledge/secrets-patterns.md, but the dump named hooks/scripts/ pre-edit-secrets.mjs and stated the two are different tables. Recording a source file that was never delivered is the defect class this repository already caught once in finding.schema.json. Neither severity nor disposition was supplied, so neither is invented — the source table carries a name and a pattern and nothing else. Verification log in docs/extraction-plan.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
This commit is contained in:
parent
f4aa8b66f4
commit
c3852c9148
2 changed files with 177 additions and 1 deletions
|
|
@ -49,7 +49,7 @@ data, never from recollection or inference.
|
||||||
|---|---|
|
|---|---|
|
||||||
| `lexicon/injection-lexicon.json` | `scanners/lib/injection-patterns.mjs` |
|
| `lexicon/injection-lexicon.json` | `scanners/lib/injection-patterns.mjs` |
|
||||||
| `codepoints/carriers.json` | `scanners/unicode-scanner.mjs` + `scanners/lib/string-utils.mjs` (incl. `HOMOGLYPH_MAP`) |
|
| `codepoints/carriers.json` | `scanners/unicode-scanner.mjs` + `scanners/lib/string-utils.mjs` (incl. `HOMOGLYPH_MAP`) |
|
||||||
| `signatures/secret-egress.json` | `knowledge/secrets-patterns.md` — the **18-entry hook table**, NOT the PCRE-flavored agent-consumed variant |
|
| `signatures/secret-egress.json` | `hooks/scripts/pre-edit-secrets.mjs` — `SECRET_PATTERNS`, the **18-entry hook table**, NOT the PCRE-flavored agent-consumed variant in `knowledge/secrets-patterns.md`. *(Corrected 2026-08-09: this row originally named `knowledge/secrets-patterns.md` as the source file. The delivered dump named `pre-edit-secrets.mjs` and stated explicitly that the two are different files. The row now names the file that was actually delivered.)* |
|
||||||
| `signatures/malware-signatures.json` | `knowledge/signatures.json` (the SIG scanner) |
|
| `signatures/malware-signatures.json` | `knowledge/signatures.json` (the SIG scanner) |
|
||||||
| `signatures/active-content.json` | currently only in a guard repo's `active_content.py`. If unavailable: stub with a version field and a TODO naming the source |
|
| `signatures/active-content.json` | currently only in a guard repo's `active_content.py`. If unavailable: stub with a version field and a TODO naming the source |
|
||||||
| `calibration/calibration.json` | `scanners/lib/severity.mjs` — thresholds + scanner caps |
|
| `calibration/calibration.json` | `scanners/lib/severity.mjs` — thresholds + scanner caps |
|
||||||
|
|
@ -164,6 +164,33 @@ have no entry in the fold map. The dump states the presence set and the fold map
|
||||||
deliberately distinct. The U+0456 / U+0445 divergence is reported to `llm-security` rather
|
deliberately distinct. The U+0456 / U+0445 divergence is reported to `llm-security` rather
|
||||||
than fixed here.
|
than fixed here.
|
||||||
|
|
||||||
|
### `signatures/secret-egress.json` — extracted 2026-08-09
|
||||||
|
|
||||||
|
Source: `llm-security/hooks/scripts/pre-edit-secrets.mjs` (`SECRET_PATTERNS`), supplied as
|
||||||
|
operator dump 2/2 through the local coord mailbox. No commit hash accompanied it.
|
||||||
|
|
||||||
|
**The seed-source row above was wrong and has been corrected.** It named
|
||||||
|
`knowledge/secrets-patterns.md`; the dump named `hooks/scripts/pre-edit-secrets.mjs` and
|
||||||
|
stated that the two are different tables — the second is PCRE-flavoured and agent-consumed
|
||||||
|
and stays where it is. Recording a source file that was never delivered is the same defect
|
||||||
|
class as the lossiness claim corrected in `finding.schema.json`, so it is corrected here in
|
||||||
|
the same commit as the file it describes.
|
||||||
|
|
||||||
|
| Check | Method | Result |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| JSON well-formed, `version` present, LF, trailing newline | `python3 -m json.tool` + byte scan | pass |
|
||||||
|
| Pattern text and flags reproduce the source | rebuilt the table **from the commons JSON alone**, sorted by the declared `order`, and diffed name, `.source` and `.flags` against the imported dump module | 18/18, 0 differences, **18/18 byte-identical** — no normalisation needed |
|
||||||
|
| Every pattern compiles in both runtimes | `new RegExp` bare and under `u` in Node; `re.compile` with `re.I` where declared in Python | 18/18 in all three modes, 0 failures |
|
||||||
|
| The ordering contract holds, and is not decorative | reproduced first-match labelling from the commons order for a Bearer header containing a JWT and for a bare JWT, against the source table | both labels identical to source: header case → `Authorization header with token`, bare case → `JWT (three-part token)` |
|
||||||
|
| Reordering is detectable, not silent | ran the same Bearer input through a reversed table | label changes to `JWT (three-part token)` — order is load-bearing, which is why every entry carries an explicit `order` field |
|
||||||
|
| `order` is contiguous | compared to `range(18)` | 0–17, no gaps |
|
||||||
|
|
||||||
|
Not verified, and not claimed: that the dump matches the module. Not supplied, and therefore
|
||||||
|
not invented: any severity or per-entry disposition — the source table carries a name and a
|
||||||
|
pattern and nothing else. Out of scope by the dump's own statement: the runtime
|
||||||
|
policy-injected custom patterns (entries 19+). A consumer matching only this table matches
|
||||||
|
**less** than the seed hook does when a policy is loaded.
|
||||||
|
|
||||||
## Definition of done for v0.1.0
|
## Definition of done for v0.1.0
|
||||||
|
|
||||||
1. Repository initialized, Forgejo remote `open/llm-security-commons`, MIT, `STATE.md`
|
1. Repository initialized, Forgejo remote `open/llm-security-commons`, MIT, `STATE.md`
|
||||||
|
|
|
||||||
149
signatures/secret-egress.json
Normal file
149
signatures/secret-egress.json
Normal file
|
|
@ -0,0 +1,149 @@
|
||||||
|
{
|
||||||
|
"version": "0.1.0",
|
||||||
|
"id": "secret-egress",
|
||||||
|
"description": "Credential and token shapes that must never leave a machine: the fixed pattern table a pre-write guard matches against content before it is persisted. Detection data only - what to DO when one matches (block, warn, redact) is the consumer's policy and is not described here.",
|
||||||
|
"owasp": "LLM02",
|
||||||
|
"match_semantics": "first match wins; patterns are evaluated in ascending `order`",
|
||||||
|
"$comment": "Extracted without behaviour change from llm-security/hooks/scripts/pre-edit-secrets.mjs (`SECRET_PATTERNS`), delivered as operator dump 2/2 through the local coord mailbox on 2026-08-09. NOTE THE SOURCE FILE: the dump states explicitly that this is the engine-consumed hook table and NOT knowledge/secrets-patterns.md, which is a separate PCRE-flavoured agent-consumed variant that stays where it is. This repository's own extraction plan originally named the wrong one of the two; the file recorded here is the one that was actually delivered. Only the 18 fixed entries are data - the dump states that entries 19 and beyond are policy-injected custom patterns at runtime and are not part of the base table.",
|
||||||
|
"provenance": {
|
||||||
|
"source_repo": "llm-security",
|
||||||
|
"source_files": [
|
||||||
|
"hooks/scripts/pre-edit-secrets.mjs"
|
||||||
|
],
|
||||||
|
"source_exports": [
|
||||||
|
"SECRET_PATTERNS"
|
||||||
|
],
|
||||||
|
"source_delivery": "operator dump 2/2, coord message from llm-security, 2026-08-09",
|
||||||
|
"source_commit": "unknown - not supplied with the dump",
|
||||||
|
"verified": "differentially, against the dump",
|
||||||
|
"evidence_limits": [
|
||||||
|
"The dump is a transcription of the source module, not the module file itself. The checks recorded for this file prove that this JSON agrees with the DUMP; dump-to-module fidelity is llm-security's assertion, not a result reproduced here.",
|
||||||
|
"No severity, and no per-entry disposition, was supplied. The source table carries a name and a pattern and nothing else, so neither is invented here.",
|
||||||
|
"The runtime-injected custom patterns (entries 19+) are policy, not data, and are out of scope. A consumer that matches only this table matches LESS than the seed hook does when a policy is loaded."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"ordering": {
|
||||||
|
"normative": true,
|
||||||
|
"$comment": "Array order is part of the contract, not an artefact of serialisation. The source places 'JWT (three-part token)' last deliberately, so that a token inside an Authorization header is reported as 'Authorization header with token' rather than as a bare JWT. A consumer that reorders this table, or that reports all matches instead of the first, will label the same input differently from the seed runtime even though both detected it. The explicit `order` field on every entry exists so that reordering cannot happen silently through a JSON round-trip.",
|
||||||
|
"last_entry_is_load_bearing": "JWT (three-part token)"
|
||||||
|
},
|
||||||
|
"dialect": {
|
||||||
|
"name": "ecmascript",
|
||||||
|
"$comment": "Patterns are ECMAScript regular-expression source text exactly as the source literals spell it. Flags are declared per pattern; an entry with no `flags` key carries no flags. All 18 compile in Node with their declared flags, in Node with `u` added, and in Python `re` with the equivalent re.I.",
|
||||||
|
"flags": {
|
||||||
|
"i": "case-insensitive"
|
||||||
|
},
|
||||||
|
"features_used": [
|
||||||
|
"non-capturing groups: (?:...)",
|
||||||
|
"bounded quantifiers: {n,m}",
|
||||||
|
"word boundaries: \\b",
|
||||||
|
"character classes"
|
||||||
|
],
|
||||||
|
"translation_notes": [
|
||||||
|
"Python (`re`): compile with re.I where flags contain `i`. No rewriting needed; verified by compiling all 18.",
|
||||||
|
"Two patterns contain `\\/` - the redundant escape a JavaScript regex LITERAL requires and that `RegExp.prototype.source` preserves ('Slack/Discord Webhook URL' and 'Database connection string'). Kept byte-identical because Node bare, Node under `u` and Python `re` all accept it. Engines that reject unknown escapes (Go `regexp`, RE2) MUST report these two as unsupported rather than skip them silently.",
|
||||||
|
"The 'Generic credential assignment' and 'Authorization header with token' entries are shape matches, not proofs of a live credential. A consumer treating every match as a confirmed leak will produce false positives; that trade-off belongs to the consumer's policy, not to this table."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"normalisations": [],
|
||||||
|
"normalisations_note": "Empty by result, not by omission: all 18 patterns are byte-identical to the source, verified below. No escaping change was needed.",
|
||||||
|
"patterns": [
|
||||||
|
{
|
||||||
|
"order": 0,
|
||||||
|
"name": "AWS Access Key ID",
|
||||||
|
"pattern": "AKIA[0-9A-Z]{16}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 1,
|
||||||
|
"name": "AWS Secret Access Key",
|
||||||
|
"pattern": "(?:aws_secret(?:_access)?_key|AWS_SECRET(?:_ACCESS)?_KEY)\\s*[=:]\\s*['\"]?[0-9a-zA-Z/+=]{40}['\"]?",
|
||||||
|
"flags": "i"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 2,
|
||||||
|
"name": "Azure Connection String (AccountKey/SharedAccessKey/sig)",
|
||||||
|
"pattern": "(?:AccountKey|SharedAccessKey|sig)=[A-Za-z0-9+/=]{20,}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 3,
|
||||||
|
"name": "Azure AD ClientSecret",
|
||||||
|
"pattern": "(?:client[_-]?secret|ClientSecret)\\s*[=:]\\s*['\"][^'\"]{8,}['\"]",
|
||||||
|
"flags": "i"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 4,
|
||||||
|
"name": "Azure AI Services Key",
|
||||||
|
"pattern": "Ocp-Apim-Subscription-Key\\s*[=:]\\s*['\"]?[0-9a-f]{32}['\"]?",
|
||||||
|
"flags": "i"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 5,
|
||||||
|
"name": "GitHub Token",
|
||||||
|
"pattern": "(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9_]{36,}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 6,
|
||||||
|
"name": "npm Token",
|
||||||
|
"pattern": "npm_[A-Za-z0-9]{36}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 7,
|
||||||
|
"name": "Anthropic API Key",
|
||||||
|
"pattern": "\\bsk-ant-api03-[A-Za-z0-9_-]{93}\\b"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 8,
|
||||||
|
"name": "OpenAI Project Key",
|
||||||
|
"pattern": "\\bsk-proj-[A-Za-z0-9_-]{40,}\\b"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 9,
|
||||||
|
"name": "GitHub Fine-Grained PAT",
|
||||||
|
"pattern": "\\bgithub_pat_[A-Za-z0-9_]{82}\\b"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 10,
|
||||||
|
"name": "Google API Key",
|
||||||
|
"pattern": "\\bAIza[0-9A-Za-z_-]{35}\\b"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 11,
|
||||||
|
"name": "Private Key PEM Block",
|
||||||
|
"pattern": "-----BEGIN (?:RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 12,
|
||||||
|
"name": "JWT Secret",
|
||||||
|
"pattern": "JWT[_-]?SECRET\\s*[=:]\\s*['\"][^'\"]{8,}['\"]",
|
||||||
|
"flags": "i"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 13,
|
||||||
|
"name": "Slack/Discord Webhook URL",
|
||||||
|
"pattern": "https:\\/\\/(?:hooks\\.slack\\.com\\/services|discord(?:app)?\\.com\\/api\\/webhooks)\\/"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 14,
|
||||||
|
"name": "Generic credential assignment",
|
||||||
|
"pattern": "(?:password|passwd|secret|token|api[_-]?key)\\s*[=:]\\s*['\"][^'\"]{8,}['\"]",
|
||||||
|
"flags": "i"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 15,
|
||||||
|
"name": "Authorization header with token",
|
||||||
|
"pattern": "[Bb]earer [A-Za-z0-9\\-._~+/]{20,}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 16,
|
||||||
|
"name": "Database connection string",
|
||||||
|
"pattern": "(?:postgres|mysql|mongodb|redis):\\/\\/[^\\s]+@[^\\s]+",
|
||||||
|
"flags": "i"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"order": 17,
|
||||||
|
"name": "JWT (three-part token)",
|
||||||
|
"pattern": "\\beyJ[A-Za-z0-9_-]{10,}\\.[A-Za-z0-9_-]{10,}\\.[A-Za-z0-9_-]{10,}\\b"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"count": 18
|
||||||
|
}
|
||||||
Loading…
Add table
Add a link
Reference in a new issue