open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

feat(signatures): the staleness we disclosed is closed by reading the module, not the message

Browse source 
secret-egress.json 0.2.0 -> 0.3.0. `OpenAI Legacy API Key` enters at order 17,
second to last; JWT stays last because ordering.last_entry_is_load_bearing says
it must. 18 -> 19.

The regex was in the coord message that reported it. That is the path
evidence_limits explicitly ruled out, so it was read out of the module text at
a pinned public commit instead: refs/heads/main = 47905da, and 088e458 (which
carries the entry) confirmed an ancestor with `git merge-base --is-ancestor`
rather than accepted from their log.

The entry is the smaller half. All 19 positions were compared against the
module - name, source, flags, order - with 0 divergences, so positions 0-16 are
no longer resting on a 2026-08-09 transcription whose module fidelity stood
recorded as llm-security's assertion. It is reproduced now, and both the
fidelity bullet and the staleness bullet retire.

manifest.json 0.3.2: the blocker prose promised its note would stand until this
landed. Item (2) is marked closed and the count moves 18 -> 19. The blocker
itself does NOT close - 19 against the guard's 25 at different cut points is a
table reconciliation nobody has performed, and one closed hole is not that.

Verified: JSON well-formed, orders contiguous 0..18, count matches array length,
new pattern compiles in Node bare, Node `u` and Python `re`, and does not match
sk-ant-/sk-proj- shapes. Charter clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLEZ4XCSnSrQUFA8SzkQB4
This commit is contained in:
Kjell Tore Guttormsen 2026-08-11 21:52:24 +02:00
commit d467324380
2 changed files with 20 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

4
conformance/manifest.json
View file

@ -1,5 +1,5 @@
{
"version": "0.3.1",
"version": "0.3.2",
"id": "llm-security-commons/conformance",
"description": "Enumeration and measurement header for the conformance corpus. Every case directory holds input.txt (the exact bytes to scan) and expected.json (the findings a conforming runtime must produce). The normative reading of those files is spec/conformance-corpus.md; this file records where the cases came from and what was measured.",
"$comment": "Fixture files carry no individual version field. The corpus is versioned as a whole, here — a case is added, removed or corrected by bumping this version, and a case-id change is a MAJOR bump because consumers name cases.",
@ -29,7 +29,7 @@
"signatures/secret-egress.json": 1,
"blockers": {
"codepoints/carriers.json": "No adoptable id space, and a second problem underneath it. The guard emits TWO stage-coupled labels for the same carrier depending on pipeline position — `sanitize:zero-width` (input) versus `output:zero-width-present` (output), and the same split for bidi and unicode-tag; llm-security emits prose titles (unicode-scanner.mjs:191,236). A commons id would therefore have to be invented stage-neutral, which no other id space here required. And because `exact-within-scope` compares a finding SET, a commons id aliasing both guard labels would make the verdict depend on which entry point the runtime was measured through — an entry-point dependence the lexicon cases do not have, since this manifest pins entry point as a measurement fact rather than as contract.",
"signatures/secret-egress.json": "Not an id-naming question at all. The two runtimes carry DIFFERENT TABLES, not two namings of one: this file holds 18 entries from llm-security, the guard's `_SECRET_PATTERNS` (output.py) holds 25 at different cut points — this file's single `GitHub Token` is four ids there, `Private Key PEM Block` is three, `Database connection string` is four. `aws-access-key-id` is the one clean one-to-one, which is why exactly one egress case was ever offered. A shared id space presupposes a table reconciliation that has not happened. Membership diverges both ways, and `Slack/Discord Webhook URL` (order 13) and `Azure AI Services Key` (order 4) are here and absent from the guard. CORRECTED IN 0.3.1: through 0.3.0 this text ended '(the guard has `gcp-service-account-json` and `openai-api-key-legacy`, which are absent here; ...)'. Two entries were named as one kind of fact, and they are two different kinds. Measured here on 2026-08-11 by running this file's own 18 patterns, in `order`, over a service-account document, against the guard at commit `e671edb`. (1) `gcp-service-account-json` is NOT a coverage hole here. A complete service-account key file matches this table at order 11, `Private Key PEM Block`: that pattern's prefix group `(?:RSA |EC |DSA |OPENSSH )?` is optional, so the bare PKCS#8 header `-----BEGIN PRIVATE KEY-----` such a file carries is matched. What differs is the CUT POINT: the same document with its `private_key` field removed matches nothing here, while the guard's `\"type\"\\s*:\\s*\"service_account\"` still fires — the guard detects the document MARKER, this table detects the KEY MATERIAL. Both fire on a real key file; only the guard fires on a stripped one. That is exactly the cut-point divergence this blocker is about, and it is not a missing entry. (2) `openai-api-key-legacy` IS a real hole here as of this version: no pattern in this file matches a legacy `sk-…T3BlbkFJ…` key, measured the same way. llm-security reports (coord, 2026-08-11) having added `OpenAI Legacy API Key` to `SECRET_PATTERNS`, 18 -> 19 — their claim, not reproduced here, because the commit carrying it is not on their public remote yet. This file is therefore knowingly STALE against its own declared source until it can be re-extracted from a pinned commit, and this note stays until that re-extraction lands."
"signatures/secret-egress.json": "Not an id-naming question at all. The two runtimes carry DIFFERENT TABLES, not two namings of one: this file holds 19 entries from llm-security, the guard's `_SECRET_PATTERNS` (output.py) holds 25 at different cut points — this file's single `GitHub Token` is four ids there, `Private Key PEM Block` is three, `Database connection string` is four. `aws-access-key-id` is the one clean one-to-one, which is why exactly one egress case was ever offered. A shared id space presupposes a table reconciliation that has not happened. Membership diverges both ways, and `Slack/Discord Webhook URL` (order 13) and `Azure AI Services Key` (order 4) are here and absent from the guard. CORRECTED IN 0.3.1: through 0.3.0 this text ended '(the guard has `gcp-service-account-json` and `openai-api-key-legacy`, which are absent here; ...)'. Two entries were named as one kind of fact, and they are two different kinds. Measured here on 2026-08-11 by running this file's own patterns, in `order`, over a service-account document, against the guard at commit `e671edb` — 18 patterns then, 19 since `secret-egress.json` 0.3.0. (1) `gcp-service-account-json` is NOT a coverage hole here. A complete service-account key file matches this table at order 11, `Private Key PEM Block`: that pattern's prefix group `(?:RSA |EC |DSA |OPENSSH )?` is optional, so the bare PKCS#8 header `-----BEGIN PRIVATE KEY-----` such a file carries is matched. What differs is the CUT POINT: the same document with its `private_key` field removed matches nothing here, while the guard's `\"type\"\\s*:\\s*\"service_account\"` still fires — the guard detects the document MARKER, this table detects the KEY MATERIAL. Both fire on a real key file; only the guard fires on a stripped one. That is exactly the cut-point divergence this blocker is about, and it is not a missing entry. (2) `openai-api-key-legacy` WAS a real hole here and is CLOSED IN 0.3.2. Through `secret-egress.json` 0.2.0 no pattern in this file matched a legacy `sk-…T3BlbkFJ…` key, measured the same way, and llm-security's 18 -> 19 report stood recorded as their unreproduced claim because the commit carrying it was not yet on their public remote. It is now: `refs/heads/main` reads `47905da`, `088e458` is an ancestor of it (checked with `git merge-base --is-ancestor`, not read off their log), and `secret-egress.json` 0.3.0 carries the entry re-extracted from the module text at that commit, with all 19 positions compared for name, source, flags and order. So the two tables now agree on this one shape. What has NOT changed is the reason this blocker exists: 19 against 25 at different cut points is still a table reconciliation nobody has performed, and one closed hole is not that reconciliation."
}
},
"omitted_payloads": [