feat(schema): close the finding contract against its producer

The schema was seeded from sarif-formatter.mjs, which consumes findings.
That could only ever establish a lower bound on the property set, so
additionalProperties had to stay open. The producer is now readable:
finding() in scanners/lib/output.mjs line 32, at b0de0ca. It returns an
object literal with exactly ten keys and no spread, so the set is complete
and the schema closes.

Added: id and evidence, the two keys a consumer-side reading could not see.
id gets its own definition, DS-<prefix>-<counter>, with pattern
^DS-[A-Za-z]+-[0-9]{3,} - the {3,} because padStart(3) is a minimum, so a
run past 999 findings produces four digits. It comes from a process-global
counter and is stable neither across runs nor across processes; the
definition says so before someone keys on it.

Nullability is evidence now, not convention. Five keys are emitted as null
rather than omitted, so a serialised finding always carries all ten. The
four assigned straight from opts are the exception: omit description and
the key is undefined and disappears from the JSON. Reproduced against the
real producer - ten keys in memory, nine serialised.

owasp is a string, not an array, and not one code. Multiple codes are
joined with ", ". Measured across the seed runtime: 31 distinct values over
157 sites, 13 multi-code, and four that mix taxonomies inside a single
value with no discriminator. That has a consequence nobody had written
down: sarif-formatter builds tags: [f.owasp], so "LLM06, ASI02" becomes ONE
tag with a comma in it and nothing filtering on LLM06 matches. Reproduced
end to end through the real finding() and toSARIF(), logged as
known_lossiness.owasp-tag-not-split. It is consumer behaviour, not data, so
it is reported rather than fixed here.

The JSONL profile is set to "not applicable" rather than "unspecified".
The distinction carries weight: unspecified would assert a profile exists
and has merely not been written down. There is no finding-JSONL - findings
are emitted only inside one JSON envelope. The single module that does
write JSONL, audit-trail.mjs, writes audit events under a different schema
where owasp is an ARRAY. Same field name, different type, same repository.
The profile records that trap instead of leaving a TODO.

Verified: valid Draft 2020-12, every finding built by the real producer
validates, and four negative controls are rejected.

One new open question left unpatched: the producer's JSDoc lists seventeen
scanner prefixes including IDE, while all four maps in owasp-map.json are
keyed on sixteen without it. An IDE finding has no taxonomy mapping
anywhere. Adding the key would be inventing detection data.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SNMcqrfNyoLRQ7qXUFZnb9
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 22:46:13 +02:00
commit 7e2e92eecf
2 changed files with 116 additions and 32 deletions

View file

@ -18,7 +18,7 @@ change. **Not yet tagged** — see *Not included* below.
### Added
- `schema/finding.schema.json` — the finding contract plus the SARIF output profile.
Normative. The JSONL profile is deliberately left `unspecified`.
Normative. Closed against the producer in 0.2.0; the JSONL profile is `not applicable`.
- `signatures/active-content.json` — the EchoLeak class (CVE-2025-32711): 17 patterns,
severities, opacity floors and pass order, from the Python guard.
- `lexicon/injection-lexicon.json` — 83 prompt-injection patterns in four families
@ -53,6 +53,53 @@ checks that were not run instead of attaching a caveat to a pass.
### Changed
- `schema/finding.schema.json` **0.1.0 → 0.2.0** — the schema is **closed**. It was seeded
from `sarif-formatter.mjs`, which *consumes* findings, so its property list could only ever
be a lower bound and `additionalProperties` had to stay open. The producer is now known —
`finding()` in `scanners/lib/output.mjs`, line 32 — and it returns an object literal with
**exactly ten keys and no spread**: `id`, `scanner`, `severity`, `title`, `description`,
`file`, `line`, `evidence`, `owasp`, `recommendation`. `additionalProperties` is `false`,
and the two keys the old schema never knew about (`id`, `evidence`) are added.
`id` gets its own definition: `DS-<prefix>-<counter>`, pattern `^DS-[A-Za-z]+-[0-9]{3,}$`.
The `{3,}` is deliberate — `padStart(3, '0')` is a minimum, so a run emitting more than 999
findings produces four digits. The id comes from a process-global counter, so it is stable
neither across runs nor across processes, and the definition says so before someone keys on it.
Nullability is now evidence rather than convention. Five keys are emitted as `null` rather
than omitted (`opts.x || null`), so a serialised finding always carries all ten. The
exception is the four assigned straight from `opts`: omit `description` and the key is
`undefined` and vanishes from the JSON. Verified by calling the real producer — ten keys in
memory, nine after serialisation.
**`owasp` is a string, not an array, and not one code.** Multiple codes are joined with
`, `. Measured across the seed runtime: 31 distinct values over 157 emission sites, 13 of
them multi-code, and **four mix taxonomies inside a single value** (`LLM06, ASI02` and
friends) with no discriminator saying which is which. That sharpens the edition problem
`mapping/owasp-map.json` already records, and it has a consequence nobody had written down:
`sarif-formatter.mjs` builds `tags: [f.owasp]`, so a finding anchored to two taxonomies
produces **one** SARIF tag with a comma in it. Nothing filtering on `LLM06` will match.
Reproduced end to end through the real `finding()` and `toSARIF()`, and logged as
`known_lossiness.owasp-tag-not-split` — consumer behaviour in `llm-security`, not data, so
it is reported rather than fixed here.
The **JSONL profile is `not applicable`, not `unspecified`** — the distinction is the point.
`unspecified` would claim a profile exists and merely has not been written down. No
finding-JSONL exists: findings are emitted only inside a single JSON envelope
(`output.mjs:140`). The one module that does write JSONL, `audit-trail.mjs`, writes *audit
events* under a different schema — where `owasp` is an **array**. Same field name, different
type, same repository. A consumer reading both through one code path will be wrong about one
of them, so the profile records the trap instead of leaving a TODO.
Verified: the schema is valid Draft 2020-12, every finding built by the real producer
validates against it, and four negative controls (extra property, missing `id`, malformed
`id`, unknown severity) are all rejected.
One new open question, unpatched by design: the producer's JSDoc lists **seventeen** scanner
prefixes including `IDE`, while all four maps in `mapping/owasp-map.json` are keyed on
**sixteen** without it. An `IDE` finding has no taxonomy mapping in any map. Adding the key
would be inventing detection data.
- `lexicon/injection-lexicon.json` **0.4.0 → 0.5.0** — the last null in the file is filled and
the id space is ratified. Two blockers close, no detection data moves.