feat(signatures): the staleness we disclosed is closed by reading the module, not the message

secret-egress.json 0.2.0 -> 0.3.0. `OpenAI Legacy API Key` enters at order 17,
second to last; JWT stays last because ordering.last_entry_is_load_bearing says
it must. 18 -> 19.

The regex was in the coord message that reported it. That is the path
evidence_limits explicitly ruled out, so it was read out of the module text at
a pinned public commit instead: refs/heads/main = 47905da, and 088e458 (which
carries the entry) confirmed an ancestor with `git merge-base --is-ancestor`
rather than accepted from their log.

The entry is the smaller half. All 19 positions were compared against the
module - name, source, flags, order - with 0 divergences, so positions 0-16 are
no longer resting on a 2026-08-09 transcription whose module fidelity stood
recorded as llm-security's assertion. It is reproduced now, and both the
fidelity bullet and the staleness bullet retire.

manifest.json 0.3.2: the blocker prose promised its note would stand until this
landed. Item (2) is marked closed and the count moves 18 -> 19. The blocker
itself does NOT close - 19 against the guard's 25 at different cut points is a
table reconciliation nobody has performed, and one closed hole is not that.

Verified: JSON well-formed, orders contiguous 0..18, count matches array length,
new pattern compiles in Node bare, Node `u` and Python `re`, and does not match
sk-ant-/sk-proj- shapes. Charter clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLEZ4XCSnSrQUFA8SzkQB4
This commit is contained in:
Kjell Tore Guttormsen 2026-08-11 21:52:24 +02:00
commit d467324380
2 changed files with 20 additions and 14 deletions

View file

@ -1,5 +1,5 @@
{ {
"version": "0.3.1", "version": "0.3.2",
"id": "llm-security-commons/conformance", "id": "llm-security-commons/conformance",
"description": "Enumeration and measurement header for the conformance corpus. Every case directory holds input.txt (the exact bytes to scan) and expected.json (the findings a conforming runtime must produce). The normative reading of those files is spec/conformance-corpus.md; this file records where the cases came from and what was measured.", "description": "Enumeration and measurement header for the conformance corpus. Every case directory holds input.txt (the exact bytes to scan) and expected.json (the findings a conforming runtime must produce). The normative reading of those files is spec/conformance-corpus.md; this file records where the cases came from and what was measured.",
"$comment": "Fixture files carry no individual version field. The corpus is versioned as a whole, here — a case is added, removed or corrected by bumping this version, and a case-id change is a MAJOR bump because consumers name cases.", "$comment": "Fixture files carry no individual version field. The corpus is versioned as a whole, here — a case is added, removed or corrected by bumping this version, and a case-id change is a MAJOR bump because consumers name cases.",
@ -29,7 +29,7 @@
"signatures/secret-egress.json": 1, "signatures/secret-egress.json": 1,
"blockers": { "blockers": {
"codepoints/carriers.json": "No adoptable id space, and a second problem underneath it. The guard emits TWO stage-coupled labels for the same carrier depending on pipeline position — `sanitize:zero-width` (input) versus `output:zero-width-present` (output), and the same split for bidi and unicode-tag; llm-security emits prose titles (unicode-scanner.mjs:191,236). A commons id would therefore have to be invented stage-neutral, which no other id space here required. And because `exact-within-scope` compares a finding SET, a commons id aliasing both guard labels would make the verdict depend on which entry point the runtime was measured through — an entry-point dependence the lexicon cases do not have, since this manifest pins entry point as a measurement fact rather than as contract.", "codepoints/carriers.json": "No adoptable id space, and a second problem underneath it. The guard emits TWO stage-coupled labels for the same carrier depending on pipeline position — `sanitize:zero-width` (input) versus `output:zero-width-present` (output), and the same split for bidi and unicode-tag; llm-security emits prose titles (unicode-scanner.mjs:191,236). A commons id would therefore have to be invented stage-neutral, which no other id space here required. And because `exact-within-scope` compares a finding SET, a commons id aliasing both guard labels would make the verdict depend on which entry point the runtime was measured through — an entry-point dependence the lexicon cases do not have, since this manifest pins entry point as a measurement fact rather than as contract.",
"signatures/secret-egress.json": "Not an id-naming question at all. The two runtimes carry DIFFERENT TABLES, not two namings of one: this file holds 18 entries from llm-security, the guard's `_SECRET_PATTERNS` (output.py) holds 25 at different cut points — this file's single `GitHub Token` is four ids there, `Private Key PEM Block` is three, `Database connection string` is four. `aws-access-key-id` is the one clean one-to-one, which is why exactly one egress case was ever offered. A shared id space presupposes a table reconciliation that has not happened. Membership diverges both ways, and `Slack/Discord Webhook URL` (order 13) and `Azure AI Services Key` (order 4) are here and absent from the guard. CORRECTED IN 0.3.1: through 0.3.0 this text ended '(the guard has `gcp-service-account-json` and `openai-api-key-legacy`, which are absent here; ...)'. Two entries were named as one kind of fact, and they are two different kinds. Measured here on 2026-08-11 by running this file's own 18 patterns, in `order`, over a service-account document, against the guard at commit `e671edb`. (1) `gcp-service-account-json` is NOT a coverage hole here. A complete service-account key file matches this table at order 11, `Private Key PEM Block`: that pattern's prefix group `(?:RSA |EC |DSA |OPENSSH )?` is optional, so the bare PKCS#8 header `-----BEGIN PRIVATE KEY-----` such a file carries is matched. What differs is the CUT POINT: the same document with its `private_key` field removed matches nothing here, while the guard's `\"type\"\\s*:\\s*\"service_account\"` still fires — the guard detects the document MARKER, this table detects the KEY MATERIAL. Both fire on a real key file; only the guard fires on a stripped one. That is exactly the cut-point divergence this blocker is about, and it is not a missing entry. (2) `openai-api-key-legacy` IS a real hole here as of this version: no pattern in this file matches a legacy `sk-…T3BlbkFJ…` key, measured the same way. llm-security reports (coord, 2026-08-11) having added `OpenAI Legacy API Key` to `SECRET_PATTERNS`, 18 -> 19 — their claim, not reproduced here, because the commit carrying it is not on their public remote yet. This file is therefore knowingly STALE against its own declared source until it can be re-extracted from a pinned commit, and this note stays until that re-extraction lands." "signatures/secret-egress.json": "Not an id-naming question at all. The two runtimes carry DIFFERENT TABLES, not two namings of one: this file holds 19 entries from llm-security, the guard's `_SECRET_PATTERNS` (output.py) holds 25 at different cut points — this file's single `GitHub Token` is four ids there, `Private Key PEM Block` is three, `Database connection string` is four. `aws-access-key-id` is the one clean one-to-one, which is why exactly one egress case was ever offered. A shared id space presupposes a table reconciliation that has not happened. Membership diverges both ways, and `Slack/Discord Webhook URL` (order 13) and `Azure AI Services Key` (order 4) are here and absent from the guard. CORRECTED IN 0.3.1: through 0.3.0 this text ended '(the guard has `gcp-service-account-json` and `openai-api-key-legacy`, which are absent here; ...)'. Two entries were named as one kind of fact, and they are two different kinds. Measured here on 2026-08-11 by running this file's own patterns, in `order`, over a service-account document, against the guard at commit `e671edb` — 18 patterns then, 19 since `secret-egress.json` 0.3.0. (1) `gcp-service-account-json` is NOT a coverage hole here. A complete service-account key file matches this table at order 11, `Private Key PEM Block`: that pattern's prefix group `(?:RSA |EC |DSA |OPENSSH )?` is optional, so the bare PKCS#8 header `-----BEGIN PRIVATE KEY-----` such a file carries is matched. What differs is the CUT POINT: the same document with its `private_key` field removed matches nothing here, while the guard's `\"type\"\\s*:\\s*\"service_account\"` still fires — the guard detects the document MARKER, this table detects the KEY MATERIAL. Both fire on a real key file; only the guard fires on a stripped one. That is exactly the cut-point divergence this blocker is about, and it is not a missing entry. (2) `openai-api-key-legacy` WAS a real hole here and is CLOSED IN 0.3.2. Through `secret-egress.json` 0.2.0 no pattern in this file matched a legacy `sk-…T3BlbkFJ…` key, measured the same way, and llm-security's 18 -> 19 report stood recorded as their unreproduced claim because the commit carrying it was not yet on their public remote. It is now: `refs/heads/main` reads `47905da`, `088e458` is an ancestor of it (checked with `git merge-base --is-ancestor`, not read off their log), and `secret-egress.json` 0.3.0 carries the entry re-extracted from the module text at that commit, with all 19 positions compared for name, source, flags and order. So the two tables now agree on this one shape. What has NOT changed is the reason this blocker exists: 19 against 25 at different cut points is still a table reconciliation nobody has performed, and one closed hole is not that reconciliation."
} }
}, },
"omitted_payloads": [ "omitted_payloads": [

View file

@ -1,10 +1,10 @@
{ {
"version": "0.2.0", "version": "0.3.0",
"id": "secret-egress", "id": "secret-egress",
"description": "Credential and token shapes that must never leave a machine: the fixed pattern table a pre-write guard matches against content before it is persisted. Detection data only - what to DO when one matches (block, warn, redact) is the consumer's policy and is not described here.", "description": "Credential and token shapes that must never leave a machine: the fixed pattern table a pre-write guard matches against content before it is persisted. Detection data only - what to DO when one matches (block, warn, redact) is the consumer's policy and is not described here.",
"owasp": "LLM02", "owasp": "LLM02",
"match_semantics": "first match wins; patterns are evaluated in ascending `order`", "match_semantics": "first match wins; patterns are evaluated in ascending `order`",
"$comment": "Extracted without behaviour change from llm-security/hooks/scripts/pre-edit-secrets.mjs (`SECRET_PATTERNS`), delivered as operator dump 2/2 through the local coord mailbox on 2026-08-09. NOTE THE SOURCE FILE: the dump states explicitly that this is the engine-consumed hook table and NOT knowledge/secrets-patterns.md, which is a separate PCRE-flavoured agent-consumed variant that stays where it is. This repository's own extraction plan originally named the wrong one of the two; the file recorded here is the one that was actually delivered. Only the 18 fixed entries are data - the dump states that entries 19 and beyond are policy-injected custom patterns at runtime and are not part of the base table.", "$comment": "Extracted without behaviour change from llm-security/hooks/scripts/pre-edit-secrets.mjs (`SECRET_PATTERNS`). NOTE THE SOURCE FILE: this is the engine-consumed hook table and NOT knowledge/secrets-patterns.md, which is a separate PCRE-flavoured agent-consumed variant that stays where it is. This repository's own extraction plan originally named the wrong one of the two; the file recorded here is the one that was actually delivered. Only the 19 fixed entries are data - entries 20 and beyond are policy-injected custom patterns at runtime and are not part of the base table. Version 0.3.0 re-extracted the table from a pinned public commit rather than from a transcription: positions 0-16 were confirmed byte-identical to what the 0.1.0 dump had already produced, and order 17 (`OpenAI Legacy API Key`) was read out of the module at that commit. The one-entry staleness disclosed in 0.2.0 is therefore closed by measurement, not by transcribing the regex out of the coord message that reported it.",
"provenance": { "provenance": {
"source_repo": "llm-security", "source_repo": "llm-security",
"source_files": [ "source_files": [
@ -13,14 +13,15 @@
"source_exports": [ "source_exports": [
"SECRET_PATTERNS" "SECRET_PATTERNS"
], ],
"source_delivery": "operator dump 2/2, coord message from llm-security, 2026-08-09", "source_delivery": "0.1.0: operator dump 2/2, coord message from llm-security, 2026-08-09. 0.3.0: re-extracted from the module text at a pinned public commit, obtained with `git show <commit>:hooks/scripts/pre-edit-secrets.mjs` against the public remote.",
"source_commit": "unknown - not supplied with the dump", "source_commit": "47905dacae8bd5613c8ed76c088cb3de93f40091",
"verified": "differentially, against the dump", "source_commit_note": "`refs/heads/main` on the public remote when 0.3.0 was extracted. The commit that introduced order 17 is `088e458`, confirmed here to be an ancestor of the pinned head with `git merge-base --is-ancestor`, rather than accepted from the reporting message.",
"verified": "positionally against the module at the pinned commit: name, regex source, flags and order compared for all 19 entries, 0 divergences",
"evidence_limits": [ "evidence_limits": [
"The dump is a transcription of the source module, not the module file itself. The checks recorded for this file prove that this JSON agrees with the DUMP; dump-to-module fidelity is llm-security's assertion, not a result reproduced here.",
"No severity, and no per-entry disposition, was supplied. The source table carries a name and a pattern and nothing else, so neither is invented here.", "No severity, and no per-entry disposition, was supplied. The source table carries a name and a pattern and nothing else, so neither is invented here.",
"The runtime-injected custom patterns (entries 19+) are policy, not data, and are out of scope. A consumer that matches only this table matches LESS than the seed hook does when a policy is loaded.", "The runtime-injected custom patterns (entries 20+) are policy, not data, and are out of scope. A consumer that matches only this table matches LESS than the seed hook does when a policy is loaded.",
"STALE AGAINST ITS SOURCE, disclosed 2026-08-11 in version 0.2.0. llm-security reports having taken the source `SECRET_PATTERNS` from 18 to 19 fixed entries by adding `OpenAI Legacy API Key`. That is their report, NOT reproduced here: the commit carrying it is not on their public remote, which stood at `b1ba1fb` when this was checked. Measured here against the 18 patterns below: no legacy `sk-…T3BlbkFJ…` key shape matches any of them. So a consumer vendoring this file today under-matches the seed hook by one entry, on a live credential shape, and that is a false negative rather than a difference of opinion. It will be closed by RE-EXTRACTION from a pinned public commit — never by authoring the entry here from a coord message, which is what the behaviour-preservation rule in CLAUDE.md forbids." "Fidelity is asserted against ONE commit. The pinned commit above is where this table was read; it says nothing about any later state of the source module. A consumer that needs to know whether it has drifted must re-measure against the remote, not re-read this field.",
"The comparison covers what the table declares - name, pattern source, flags, order. It does not cover the surrounding module: how the guard reads content, what it does on a match, or which paths it exempts are engine behaviour and are deliberately absent from this file."
] ]
}, },
"ordering": { "ordering": {
@ -30,7 +31,7 @@
}, },
"dialect": { "dialect": {
"name": "ecmascript", "name": "ecmascript",
"$comment": "Patterns are ECMAScript regular-expression source text exactly as the source literals spell it. Flags are declared per pattern; an entry with no `flags` key carries no flags. All 18 compile in Node with their declared flags, in Node with `u` added, and in Python `re` with the equivalent re.I.", "$comment": "Patterns are ECMAScript regular-expression source text exactly as the source literals spell it. Flags are declared per pattern; an entry with no `flags` key carries no flags. All 19 compile in Node with their declared flags, in Node with `u` added, and in Python `re` with the equivalent re.I.",
"flags": { "flags": {
"i": "case-insensitive" "i": "case-insensitive"
}, },
@ -41,13 +42,13 @@
"character classes" "character classes"
], ],
"translation_notes": [ "translation_notes": [
"Python (`re`): compile with re.I where flags contain `i`. No rewriting needed; verified by compiling all 18.", "Python (`re`): compile with re.I where flags contain `i`. No rewriting needed; verified by compiling all 19.",
"Two patterns contain `\\/` - the redundant escape a JavaScript regex LITERAL requires and that `RegExp.prototype.source` preserves ('Slack/Discord Webhook URL' and 'Database connection string'). Kept byte-identical because Node bare, Node under `u` and Python `re` all accept it. Engines that reject unknown escapes (Go `regexp`, RE2) MUST report these two as unsupported rather than skip them silently.", "Two patterns contain `\\/` - the redundant escape a JavaScript regex LITERAL requires and that `RegExp.prototype.source` preserves ('Slack/Discord Webhook URL' and 'Database connection string'). Kept byte-identical because Node bare, Node under `u` and Python `re` all accept it. Engines that reject unknown escapes (Go `regexp`, RE2) MUST report these two as unsupported rather than skip them silently.",
"The 'Generic credential assignment' and 'Authorization header with token' entries are shape matches, not proofs of a live credential. A consumer treating every match as a confirmed leak will produce false positives; that trade-off belongs to the consumer's policy, not to this table." "The 'Generic credential assignment' and 'Authorization header with token' entries are shape matches, not proofs of a live credential. A consumer treating every match as a confirmed leak will produce false positives; that trade-off belongs to the consumer's policy, not to this table."
] ]
}, },
"normalisations": [], "normalisations": [],
"normalisations_note": "Empty by result, not by omission: all 18 patterns are byte-identical to the source, verified below. No escaping change was needed.", "normalisations_note": "Empty by result, not by omission: all 19 patterns are byte-identical to the source module at the pinned commit, compared positionally. No escaping change was needed.",
"patterns": [ "patterns": [
{ {
"order": 0, "order": 0,
@ -142,9 +143,14 @@
}, },
{ {
"order": 17, "order": 17,
"name": "OpenAI Legacy API Key",
"pattern": "\\bsk-[A-Za-z0-9]{20}T3BlbkFJ[A-Za-z0-9]{20}\\b"
},
{
"order": 18,
"name": "JWT (three-part token)", "name": "JWT (three-part token)",
"pattern": "\\beyJ[A-Za-z0-9_-]{10,}\\.[A-Za-z0-9_-]{10,}\\.[A-Za-z0-9_-]{10,}\\b" "pattern": "\\beyJ[A-Za-z0-9_-]{10,}\\.[A-Za-z0-9_-]{10,}\\.[A-Za-z0-9_-]{10,}\\b"
} }
], ],
"count": 18 "count": 19
} }