open/llm-security-commons
1
0
Fork 0
Code Issues Releases 1 Activity

feat(schema): add finding.schema.json with normative SARIF profile

Browse source 
The finding contract plus the SARIF 2.1.0 output profile, modelled on
llm-security's sarif-formatter.mjs (operator dump). Draft 2020-12, valid
against the metaschema.

The SARIF profile is proven, not transcribed: the mapping was re-implemented
from the commons JSON alone and diffed against the real toSARIF over 10
envelope shapes (all severities incl. unknown/undefined, slug edge cases,
rule-id collision, every optional-field combination, multi-scanner, explicit
version) — 0 differences.

Three things left honest rather than closed:
- additionalProperties stays open. The formatter CONSUMES findings, so fields
  it ignores are invisible in this evidence; the property list is a lower
  bound until the producer is supplied.
- The JSONL profile is status=unspecified with a TODO. "One finding per line"
  is inference, and a guessed normative contract is worse than a missing one.
- Node and the Python guard share only severity+owasp; the rest of the field
  names diverge (scanner/title vs detector/label). Recorded as an open
  question for v0.2.0, not settled silently here.

One published claim was wrong and is corrected: rule-id collision does NOT
cover punctuation, only case and whitespace runs. Caught by executing the
claim instead of asserting it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0191AKc2qW6tmXDFSx1xn53q
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 14:52:53 +02:00
commit 0433240b14
2 changed files with 230 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

18
docs/extraction-plan.md
View file

@ -93,6 +93,24 @@ Not verified, and not claimed: that the Node consumer's active-content behaviour
this table. The source module states the Node port shares its severities; that is the
module's claim, and confirming it needs the Node file.
### `schema/finding.schema.json` — extracted 2026-08-09
Source: `llm-security/scanners/lib/sarif-formatter.mjs`, supplied as an operator dump. No
commit hash accompanied it, so provenance is recorded as `unknown` rather than guessed.
| Check | Method | Result |
| --- | --- | --- |
| JSON well-formed | `python3 -m json.tool` | pass |
| Valid JSON Schema | `jsonschema` `check_schema` against draft 2020-12 | pass |
| Accepts/rejects findings correctly | 2 valid + 3 invalid findings (missing `scanner`, unknown severity, `line: 0`) | 5/5 as intended |
| SARIF profile reproduces the source | re-implemented the mapping **from the commons JSON alone** and diffed `JSON.stringify` against the real `toSARIF` over 10 envelope shapes: empty, missing `scanners`, empty `scanners`, scanner with no findings, all five severities plus an unknown and an `undefined` one, five slug edge cases (double space, tab, newline, leading/trailing space, mixed case), a rule-id collision, all seven optional-field combinations, two scanners, and an explicit `version` argument | 10/10 identical, 0 differences |
| The three `known_lossiness` claims are true | executed each against the real formatter | all three confirmed, **and one earlier claim corrected**: punctuation does *not* collapse — the slug lowercases and collapses whitespace only, so `Zero-width carrier` and `Zero-width carrier!` remain distinct ids. The wrong claim was published in the first draft of this file and fixed before commit. |
Not verified, and recorded in the file as open: the finding **producer** was not supplied, so
the property list is a lower bound; `scanner` and `severity` are required by design rather
than by evidence; and the JSONL profile is left explicitly `unspecified` rather than
invented, because "one finding per line" is inference.
## Definition of done for v0.1.0
1. Repository initialized, Forgejo remote `open/llm-security-commons`, MIT, `STATE.md`