open/llm-security
1
0
Fork 0
Code Issues Releases 1 Activity

Merge commit 'bbd03f9b52'

Browse source
This commit is contained in:
Kjell Tore Guttormsen 2026-08-11 14:03:23 +02:00
commit 2689df09a8
9 changed files with 585 additions and 25 deletions
Download patch file Download diff file Expand all files Collapse all files

16
scanners/commons/spec/conformance-corpus.md
View file

@ -48,6 +48,22 @@ Two constraints keep the verdict from becoming an exit:
MUST run every case scoped to it, including the ones it fails. Per-case opt-out is exactly
the silent skip §1 forbids.
**The shape of the declaration.** A runtime publishing a declaration MUST use the shape in
[`schema/conformance-declaration.schema.json`](../schema/conformance-declaration.schema.json):
runtime, the commons commit measured, the declared table set, and the four verdict counts with
their total. The requirement is the shape, not a validation step — nothing in this repository
runs, so no runtime is gated on it. It exists because a free-form declaration makes
`83 passed, 6 not-applicable` unparseable by anyone but its author, and a result only its
author can read is not a cross-runtime measurement. That schema also carries the two arithmetic
invariants this section implies but cannot state in prose without ambiguity: the counts MUST sum
to the total, and the total MUST equal the corpus case count at the commit measured.
A runtime SHOULD derive its declared set from the same constant its runner uses to accept or
reject a `scope`, rather than maintain a second list beside it, and SHOULD record which of the
two it did. The anti-narrowing constraint above is then structural rather than promised:
narrowing the declaration is narrowing what the runtime accepts, which is a change to code with
tests behind it instead of an edit to a published file nobody re-reads.
**What the verdict does not say.** `not-applicable` records that a runtime does not implement
a named commons table. It is not a statement that the runtime is blind to the phenomenon
behind it — a runtime may well detect the same construct through some other mechanism that