open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity
llm-security-commons/conformance
History
Kjell Tore Guttormsen 0e765a02eb docs(security): the attack surface here is data, so the report route had to say where a wrong entry gets fixed 
org-ops recorded SECURITY.md as missing against the org standard (coord,
2026-08-11) and this repository owed it for a sharper reason than "given what
the repo is about": nothing here runs, so a report is never a crash — it is a
detection entry that looks like it works and is not looking.

SECURITY.md therefore answers what an ordinary policy does not have to: how to
report that a detection-table entry is WRONG, and why a confirmed defect in
extracted data is decided in the runtime it was extracted from before it is
changed here. Correcting it here would make the copy disagree with the
implementation it was taken from — two runtimes, two answers on one input, the
exact failure this repository exists to prevent. Two classes skip that routing:
a real secret in the history, and data authored here rather than extracted.
Fix latency is stated plainly as bounded by the owning runtime's schedule and
the consumer's pull, not by ours.

secret-egress 0.1.0 -> 0.2.0 is a staleness DISCLOSURE, not a data change: all
18 patterns byte-identical, one evidence_limits entry added. llm-security
reports the source table at 19 entries now; recorded as their report and not
reproduced, because the commit carrying it is not on their public remote —
measured at b1ba1fb today. What was measured here: none of the 18 patterns
matches a legacy sk-...T3BlbkFJ... shape. A consumer vendoring this file
under-matches the seed hook by one entry, and now reads that in the file.

manifest 0.3.0 -> 0.3.1 corrects the secret-egress blocker. Through 0.3.0 it
named gcp-service-account-json and openai-api-key-legacy together as ids
"absent here". Measured against the guard at e671edb by running this file's own
18 patterns over a service-account document: a COMPLETE service-account key
file is matched here at order 11, since the PEM entry's prefix group is
optional and the bare PKCS#8 header matches; the same document with private_key
removed matches nothing here while the guard's marker still fires. That is a
cut-point difference, which is what the blocker is about, not a missing entry.
openai-api-key-legacy IS a real hole and is now recorded as one. Folded into
the existing blocker string rather than a sibling key, because blockers is a
map from table path to text.

Verified: all JSON well-formed; every non-fixture JSON has a top-level version;
charter clean (no executable code); patterns[] and count byte-identical to HEAD
for secret-egress; manifest key set unchanged and count still 90; 90 case
directories untouched; every spec still carries its normative marker.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T85QqeiWBEoBMjWaMiBnxD
2026-08-11 14:03:24 +02:00
..
active__autolink feat(conformance): ship the six active-content cases; the id space already existed 2026-08-10 21:15:25 +02:00
active__data-uri feat(conformance): ship the six active-content cases; the id space already existed 2026-08-10 21:15:25 +02:00
active__markdown-image feat(conformance): ship the six active-content cases; the id space already existed 2026-08-10 21:15:25 +02:00
active__markdown-link feat(conformance): ship the six active-content cases; the id space already existed 2026-08-10 21:15:25 +02:00
active__raw-html feat(conformance): ship the six active-content cases; the id space already existed 2026-08-10 21:15:25 +02:00
active__reference-link feat(conformance): ship the six active-content cases; the id space already existed 2026-08-10 21:15:25 +02:00
config__bypass-safety fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
config__disable-filtering fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
config__ignore-claudemd fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
config__override-system-prompt fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
data-uri__executable fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
evasion__academic-context fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
evasion__educational fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
evasion__fictional fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
evasion__hypothetical fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
evasion__red-team fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
evasion__research-purpose fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
evasion__security-audit-sim fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
evasion__test-pretense fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
evasion__thought-experiment fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hidden-comment__agent-directed fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hitl-trap__approve-immediately fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hitl-trap__brevity-skip fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hitl-trap__details-omitted fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hitl-trap__just-approve fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hitl-trap__no-need-review fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hitl-trap__scope-minimization fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hitl-trap__suppress-review fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hitl-trap__too-long fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
homoglyph__cyrillic-latin-mix fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
html-obfuscation__aria-label fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
html-obfuscation__display-none fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
html-obfuscation__offscreen fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
html-obfuscation__visibility-hidden fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
html-obfuscation__zero-font fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
html-obfuscation__zero-opacity fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
html-obfuscation__zero-size-overflow fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hybrid-p2sql__inject-sql fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hybrid-p2sql__sql-inject fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hybrid-recursive__conditional-response fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hybrid-recursive__inject-output fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hybrid-xss__event-handler fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hybrid-xss__iframe-src fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hybrid-xss__javascript-uri fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hybrid-xss__script-tag fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
hybrid-xss__script-tag--src-no-close feat(conformance): the witness case, and the derivation rule that had no room for it 2026-08-11 13:39:19 +02:00
identity__new-directive fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
identity__new-role fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
identity__no-longer fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
identity__pretend fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
identity__you-are-now fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
indirect__direct-address fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
indirect__instruction-to-ai fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
indirect__trigger-based fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
leetspeak__disregard fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
leetspeak__forget-instructions fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
leetspeak__ignore-previous fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
leetspeak__override fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
markdown__link-anchor-injection fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
markdown__link-ref-comment fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
multi-lang__french fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
multi-lang__german fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
multi-lang__spanish fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
nl-indirection__download-execute fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
nl-indirection__extract-exfiltrate fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
nl-indirection__fetch-execute fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
nl-indirection__read-dotfile fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
nl-indirection__send-external fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
normalization__claudemd-conflict fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
normalization__prioritize-user fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
normalization__skip-confirmation fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
normalization__user-precedence fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
override__disregard fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
override__forget-instructions fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
override__ignore-previous fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
override__override-instructions fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
override__prior-constraints fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
override__suspend-constraints fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
reframing__actual-task fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
reframing__real-purpose fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
reframing__should-actually-do fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
spoofed-header__hash-system fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
spoofed-header__inst fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
spoofed-header__instructions fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
spoofed-header__pipe-system fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
spoofed-tag__system-close fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
spoofed-tag__system-open fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
sub-agent__delegate-bypass fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
sub-agent__spawn-dangerous fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
unicode__zero-width-in-word fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word 2026-08-10 04:44:59 +02:00
manifest.json docs(security): the attack surface here is data, so the report route had to say where a wrong entry gets fixed 2026-08-11 14:03:24 +02:00