fix(conformance): name the fixture field pattern_id, not id — the schema already owns that word
schema/finding.schema.json defines a finding `id` as DS-<scanner>-<counter>, built from a process-global counter: stable across neither runs nor processes, and the schema says so itself. The corpus keys its comparison on the lexicon's stable rule identity. Two normative documents in one repository using one word for both would produce runtimes failing every case for a reason unrelated to detection. Also adds spec section 3.1, which publishes the bridge a consumer actually needs and which neither normative document named: a runtime's own label reaches a pattern_id through the lexicon's `aliases` object, and a runtime absent from that object has no published way to be compared -- a mapping kept privately in a consumer is the drift this repository exists to prevent. Regenerated all 83 fixtures; re-verified from the corpus alone against both runtimes, 83 cases, 0 failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WhXDL82FRrQWedEmUg12Pj
This commit is contained in:
parent
49e1e79807
commit
b8c31d5639
85 changed files with 146 additions and 98 deletions
15
CHANGELOG.md
15
CHANGELOG.md
|
|
@ -48,9 +48,18 @@ the seven data files. Both absences are named in *Not included* rather than pape
|
||||||
manifest records the asymmetry rather than averaging it away.
|
manifest records the asymmetry rather than averaging it away.
|
||||||
|
|
||||||
- `spec/conformance-corpus.md` — **normative.** How a case is read: `input.txt` is bytes and
|
- `spec/conformance-corpus.md` — **normative.** How a case is read: `input.txt` is bytes and
|
||||||
is not to be trimmed or re-encoded, `expected.json` names findings by `id` only (severity
|
is not to be trimmed or re-encoded, `expected.json` names findings by `pattern_id` only
|
||||||
and OWASP anchor are looked up in the lexicon, never restated), and `exact-within-scope`
|
(severity and OWASP anchor are looked up in the lexicon, never restated), and
|
||||||
requires equality **restricted to the data files the case names**.
|
`exact-within-scope` requires equality **restricted to the data files the case names**.
|
||||||
|
|
||||||
|
The field is `pattern_id`, not `id`, because this repository already publishes an unrelated
|
||||||
|
finding `id`: `schema/finding.schema.json` defines it as `DS-<scanner>-<counter>` from a
|
||||||
|
process-global counter — stable across neither runs nor processes. Two normative documents
|
||||||
|
using one word for a stable rule identity and a volatile per-emission sequence number would
|
||||||
|
have produced runtimes failing every case for reasons unrelated to detection. §3.1 states
|
||||||
|
the distinction and publishes the bridge a runtime actually needs: its own label maps to a
|
||||||
|
`pattern_id` through the lexicon's `aliases` object, and a runtime absent from that object
|
||||||
|
has no published way to be compared at all.
|
||||||
|
|
||||||
Scoping is what makes exactness safe — the two runtimes do not implement the same set of
|
Scoping is what makes exactness safe — the two runtimes do not implement the same set of
|
||||||
tables, so a whole-report comparison would fail for reasons unrelated to the pattern under
|
tables, so a whole-report comparison would fail for reasons unrelated to the pattern under
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "config:bypass-safety"
|
"pattern_id": "config:bypass-safety"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "config:disable-filtering"
|
"pattern_id": "config:disable-filtering"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "config:ignore-claudemd"
|
"pattern_id": "config:ignore-claudemd"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,13 +12,13 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "config:override-system-prompt"
|
"pattern_id": "config:override-system-prompt"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "leetspeak:override"
|
"pattern_id": "leetspeak:override"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "override:override-instructions"
|
"pattern_id": "override:override-instructions"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "data-uri:executable"
|
"pattern_id": "data-uri:executable"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {
|
"observed_out_of_scope": {
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "evasion:academic-context"
|
"pattern_id": "evasion:academic-context"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "evasion:educational"
|
"pattern_id": "evasion:educational"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "evasion:fictional"
|
"pattern_id": "evasion:fictional"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "evasion:hypothetical"
|
"pattern_id": "evasion:hypothetical"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "evasion:red-team"
|
"pattern_id": "evasion:red-team"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "evasion:research-purpose"
|
"pattern_id": "evasion:research-purpose"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "evasion:security-audit-sim"
|
"pattern_id": "evasion:security-audit-sim"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "evasion:test-pretense"
|
"pattern_id": "evasion:test-pretense"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "evasion:thought-experiment"
|
"pattern_id": "evasion:thought-experiment"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hidden-comment:agent-directed"
|
"pattern_id": "hidden-comment:agent-directed"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hitl-trap:approve-immediately"
|
"pattern_id": "hitl-trap:approve-immediately"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hitl-trap:brevity-skip"
|
"pattern_id": "hitl-trap:brevity-skip"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hitl-trap:details-omitted"
|
"pattern_id": "hitl-trap:details-omitted"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hitl-trap:just-approve"
|
"pattern_id": "hitl-trap:just-approve"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hitl-trap:no-need-review"
|
"pattern_id": "hitl-trap:no-need-review"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hitl-trap:scope-minimization"
|
"pattern_id": "hitl-trap:scope-minimization"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hitl-trap:suppress-review"
|
"pattern_id": "hitl-trap:suppress-review"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hitl-trap:too-long"
|
"pattern_id": "hitl-trap:too-long"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "homoglyph:cyrillic-latin-mix"
|
"pattern_id": "homoglyph:cyrillic-latin-mix"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "html-obfuscation:aria-label"
|
"pattern_id": "html-obfuscation:aria-label"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {
|
"observed_out_of_scope": {
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "html-obfuscation:display-none"
|
"pattern_id": "html-obfuscation:display-none"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "html-obfuscation:offscreen"
|
"pattern_id": "html-obfuscation:offscreen"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "html-obfuscation:visibility-hidden"
|
"pattern_id": "html-obfuscation:visibility-hidden"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "html-obfuscation:zero-font"
|
"pattern_id": "html-obfuscation:zero-font"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "html-obfuscation:zero-opacity"
|
"pattern_id": "html-obfuscation:zero-opacity"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "html-obfuscation:zero-size-overflow"
|
"pattern_id": "html-obfuscation:zero-size-overflow"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hybrid-p2sql:inject-sql"
|
"pattern_id": "hybrid-p2sql:inject-sql"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hybrid-p2sql:sql-inject"
|
"pattern_id": "hybrid-p2sql:sql-inject"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hybrid-recursive:conditional-response"
|
"pattern_id": "hybrid-recursive:conditional-response"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hybrid-recursive:inject-output"
|
"pattern_id": "hybrid-recursive:inject-output"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hybrid-xss:event-handler"
|
"pattern_id": "hybrid-xss:event-handler"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {
|
"observed_out_of_scope": {
|
||||||
|
|
|
||||||
|
|
@ -12,10 +12,10 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hybrid-xss:iframe-src"
|
"pattern_id": "hybrid-xss:iframe-src"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "hybrid-xss:javascript-uri"
|
"pattern_id": "hybrid-xss:javascript-uri"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {
|
"observed_out_of_scope": {
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hybrid-xss:javascript-uri"
|
"pattern_id": "hybrid-xss:javascript-uri"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "hybrid-xss:script-tag"
|
"pattern_id": "hybrid-xss:script-tag"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {
|
"observed_out_of_scope": {
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "identity:new-directive"
|
"pattern_id": "identity:new-directive"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "identity:new-role"
|
"pattern_id": "identity:new-role"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "identity:no-longer"
|
"pattern_id": "identity:no-longer"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "identity:pretend"
|
"pattern_id": "identity:pretend"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "identity:you-are-now"
|
"pattern_id": "identity:you-are-now"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "indirect:direct-address"
|
"pattern_id": "indirect:direct-address"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "indirect:instruction-to-ai"
|
"pattern_id": "indirect:instruction-to-ai"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "indirect:trigger-based"
|
"pattern_id": "indirect:trigger-based"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "leetspeak:disregard"
|
"pattern_id": "leetspeak:disregard"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "leetspeak:forget-instructions"
|
"pattern_id": "leetspeak:forget-instructions"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "leetspeak:ignore-previous"
|
"pattern_id": "leetspeak:ignore-previous"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "leetspeak:override"
|
"pattern_id": "leetspeak:override"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "markdown:link-anchor-injection"
|
"pattern_id": "markdown:link-anchor-injection"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {
|
"observed_out_of_scope": {
|
||||||
|
|
|
||||||
|
|
@ -12,10 +12,10 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "markdown:link-ref-comment"
|
"pattern_id": "markdown:link-ref-comment"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "override:ignore-previous"
|
"pattern_id": "override:ignore-previous"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "multi-lang:french"
|
"pattern_id": "multi-lang:french"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "multi-lang:german"
|
"pattern_id": "multi-lang:german"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "multi-lang:spanish"
|
"pattern_id": "multi-lang:spanish"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "nl-indirection:download-execute"
|
"pattern_id": "nl-indirection:download-execute"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "nl-indirection:extract-exfiltrate"
|
"pattern_id": "nl-indirection:extract-exfiltrate"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "nl-indirection:fetch-execute"
|
"pattern_id": "nl-indirection:fetch-execute"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "nl-indirection:read-dotfile"
|
"pattern_id": "nl-indirection:read-dotfile"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "nl-indirection:send-external"
|
"pattern_id": "nl-indirection:send-external"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "normalization:claudemd-conflict"
|
"pattern_id": "normalization:claudemd-conflict"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "normalization:prioritize-user"
|
"pattern_id": "normalization:prioritize-user"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "normalization:skip-confirmation"
|
"pattern_id": "normalization:skip-confirmation"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "normalization:user-precedence"
|
"pattern_id": "normalization:user-precedence"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,10 +12,10 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "leetspeak:disregard"
|
"pattern_id": "leetspeak:disregard"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "override:disregard"
|
"pattern_id": "override:disregard"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,10 +12,10 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "leetspeak:forget-instructions"
|
"pattern_id": "leetspeak:forget-instructions"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "override:forget-instructions"
|
"pattern_id": "override:forget-instructions"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "override:ignore-previous"
|
"pattern_id": "override:ignore-previous"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,10 +12,10 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "leetspeak:override"
|
"pattern_id": "leetspeak:override"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "override:override-instructions"
|
"pattern_id": "override:override-instructions"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "override:prior-constraints"
|
"pattern_id": "override:prior-constraints"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "override:suspend-constraints"
|
"pattern_id": "override:suspend-constraints"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "reframing:actual-task"
|
"pattern_id": "reframing:actual-task"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "reframing:real-purpose"
|
"pattern_id": "reframing:real-purpose"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "reframing:should-actually-do"
|
"pattern_id": "reframing:should-actually-do"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "spoofed-header:hash-system"
|
"pattern_id": "spoofed-header:hash-system"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "spoofed-header:inst"
|
"pattern_id": "spoofed-header:inst"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "spoofed-header:instructions"
|
"pattern_id": "spoofed-header:instructions"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "spoofed-header:pipe-system"
|
"pattern_id": "spoofed-header:pipe-system"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "spoofed-tag:system-close"
|
"pattern_id": "spoofed-tag:system-close"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "spoofed-tag:system-open"
|
"pattern_id": "spoofed-tag:system-open"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "sub-agent:delegate-bypass"
|
"pattern_id": "sub-agent:delegate-bypass"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "sub-agent:spawn-dangerous"
|
"pattern_id": "sub-agent:spawn-dangerous"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {}
|
"observed_out_of_scope": {}
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [
|
"findings": [
|
||||||
{
|
{
|
||||||
"id": "unicode:zero-width-in-word"
|
"pattern_id": "unicode:zero-width-in-word"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"observed_out_of_scope": {
|
"observed_out_of_scope": {
|
||||||
|
|
|
||||||
|
|
@ -40,25 +40,64 @@ means nothing.
|
||||||
"input": { "file": "input.txt", "bytes": 24, "sha256": "…" },
|
"input": { "file": "input.txt", "bytes": 24, "sha256": "…" },
|
||||||
"scope": ["lexicon/injection-lexicon.json"],
|
"scope": ["lexicon/injection-lexicon.json"],
|
||||||
"match": "exact-within-scope",
|
"match": "exact-within-scope",
|
||||||
"findings": [{ "id": "hybrid-xss:script-tag" }],
|
"findings": [{ "pattern_id": "hybrid-xss:script-tag" }],
|
||||||
"observed_out_of_scope": { "llm_ingestion_guard": ["active:raw-html"] }
|
"observed_out_of_scope": { "llm_ingestion_guard": ["active:raw-html"] }
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
| Field | Normative? | Meaning |
|
| Field | Normative? | Meaning |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
|
| `$comment` | no | Provenance prose. Carries no contract. |
|
||||||
| `case_id` | yes | Equals the directory name. |
|
| `case_id` | yes | Equals the directory name. |
|
||||||
| `input` | yes | Integrity of the input file. See §2. |
|
| `input` | yes | Integrity of the input file. See §2. |
|
||||||
| `scope` | yes | Which commons data files this case constrains. See §4. |
|
| `scope` | yes | Which commons data files this case constrains. See §4. |
|
||||||
| `match` | yes | How `findings` is to be compared. See §4. |
|
| `match` | yes | How `findings` is to be compared. See §4. |
|
||||||
| `findings` | yes | The expected findings, named by commons pattern `id`. |
|
| `findings` | yes | The expected findings, named by commons `pattern_id`. See §3.1. |
|
||||||
| `observed_out_of_scope` | **no — informative** | What a named runtime was measured emitting outside `scope`. Evidence, never expectation. See §5. |
|
| `observed_out_of_scope` | **no — informative** | What a named runtime was measured emitting outside `scope`. Evidence, never expectation. See §5. |
|
||||||
|
|
||||||
A finding carries an `id` and nothing else. Severity, OWASP anchor and human label are
|
A finding carries a `pattern_id` and nothing else. Severity, OWASP anchor and human label
|
||||||
properties of the pattern, published once in [`lexicon/injection-lexicon.json`](../lexicon/injection-lexicon.json)
|
are properties of the pattern, published once in [`lexicon/injection-lexicon.json`](../lexicon/injection-lexicon.json)
|
||||||
and looked up there by `id`. They MUST NOT be restated in a fixture: a value written in
|
and looked up there by that id. They MUST NOT be restated in a fixture: a value written in
|
||||||
two places is a value that will eventually disagree with itself.
|
two places is a value that will eventually disagree with itself.
|
||||||
|
|
||||||
|
### 3.1 `pattern_id` is not the finding `id`
|
||||||
|
|
||||||
|
The field is called `pattern_id` and not `id` because this repository already publishes a
|
||||||
|
different `id`, and confusing the two produces a runtime that fails every case for a reason
|
||||||
|
unrelated to detection.
|
||||||
|
|
||||||
|
| | Identifies | Stable? |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `pattern_id` — here; `families[].patterns[].id` in the lexicon | the **detection rule** | Yes. A stable cross-runtime contract, ratified by both seeding runtimes. |
|
||||||
|
| `id` — in [`schema/finding.schema.json`](../schema/finding.schema.json) | one **emitted finding** in one run | No. `DS-<scanner>-<counter>`, from a process-global counter: not stable across runs, not unique across processes. The schema says so itself. |
|
||||||
|
|
||||||
|
A runtime therefore MUST NOT compare a serialized finding's `id` to a fixture's
|
||||||
|
`pattern_id`. They answer different questions.
|
||||||
|
|
||||||
|
**Getting from a runtime's own label to a `pattern_id`.** Nothing requires a runtime to name
|
||||||
|
its patterns the way this repository does, and the two seeding runtimes do not: the same rule
|
||||||
|
is `override: ignore previous instructions` in one and `override:ignore-previous` in the
|
||||||
|
other. The mapping is published, per pattern, in the lexicon's `aliases` object:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "override:ignore-previous",
|
||||||
|
"aliases": {
|
||||||
|
"llm_security": "override: ignore previous instructions",
|
||||||
|
"llm_ingestion_guard": "override:ignore-previous"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
(The lexicon's own field is spelled `id`, because that name is ratified and renaming it
|
||||||
|
would break both consumers. `pattern_id` here refers to exactly that value.)
|
||||||
|
|
||||||
|
A runtime SHOULD carry that id directly on its own pattern table, which makes the
|
||||||
|
comparison a lookup rather than a translation. A runtime that instead maps through `aliases`
|
||||||
|
MUST register its own alias there — a runtime absent from that object has no published way to
|
||||||
|
be compared, and a mapping maintained privately in a consumer is the drift this repository
|
||||||
|
exists to prevent.
|
||||||
|
|
||||||
Fixture files carry **no individual `version` field**, which is the one place this
|
Fixture files carry **no individual `version` field**, which is the one place this
|
||||||
repository's "every JSON file has a top-level `version`" convention does not apply. The
|
repository's "every JSON file has a top-level `version`" convention does not apply. The
|
||||||
corpus is versioned as a whole in [`conformance/manifest.json`](../conformance/manifest.json).
|
corpus is versioned as a whole in [`conformance/manifest.json`](../conformance/manifest.json).
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue